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SUMMARY 


PROBLEM 

1  •  In  the  past  decade  more  and  more  attention  has  been  focused 

on  the  role  of  the  human  element  weapon  systems  .  The  contribution  of 
this  personnel  element  to  system  reliability  and  effectiveness  and  the  deg¬ 
radation  in  system  performance  because  of  human  error  have  been  subjects 
of  numerous  studies  and  research  programs.  These  studies  have  substan¬ 
tiated  the  fact  that  a  significant  percentage  of  system  unreliability  is 
caused  by  human  error, 

2.  These  findings  validate  the  urgency  for  quantifying  human  per¬ 

formance  and  developing  new  techniques  for  the  prediction  and  analysis 
of  human  reliability  as  a  part  of  system  effectiveness.  It  is  virtually 
impossible  to  test  the  human  element  in  the  laboratory  under  controlled 
environment  and  to  use  the  results  for  predicting  human  performance  under 
actual  conditions.  Consequently,  data  must  be  obtained  from  the  field 
during  the  operational  and  maintenance  phases  of  weapon  systems.  This 
need  for  field  data  poses  a  major  problem  in  that  data  on  human  performance 
collected  to  date  are  of  little  consequence;  more  significantly,  no  vehicle 
exists  for  their  collection. 

As  a  result,  the  problem  of  the  data  gap  must  be  overcome  be¬ 
fore  new  techniques  for  human  reliability  analysis  can  be  developed. 

New  methods  for  acquiring  the  necessary  data  will  have  to  be  investigated, 
and  an  extensive  Navy-wide  and  preferably  DOD-wide  effort  will  have  to 
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be  initiated  to  establish  a  meaningful  data  base.  The  establishment  of 
such  a  data  base  will  require  considerable  time.  There  is  a  definite  need 
to  develop,  at  least  on  an  interim  basis,  techniques  for  human  reliability 
analysis  that  do  not  depend  on  direct  human-performance  data.  Rather, 
these  techniques  must  be  designed  so  that  by  mathematical  and/or  statis¬ 
tical  means  human  reliability  can  be  inferred  from  existing  equipment  per¬ 
formance  data. 

BACKGROUND  AND  REQUIREMENT 

4.  The  recognition  of  the  need  to  develop  new  tools  and  techniques 
for  numan  reliability  a».a lysis  has  become  widespread  throughout  the  Navy, 
manifested  by  the  issuance  of  SECNAVINST  3900.36  dated  27  January  1966. 
The  purpose  cf  this  document  is  to  "establish  policy  for  the  guidance  of 
efforts  to  increase  the  reliability  of  naval  material  and  to  assign  responsi¬ 
bility  for  its  achievement."  The  Chief  of  Naval  Personnel  is  assigned  the 
responsibility  to: 

"  1.  Coordinate  with  the  Chief  of  Naval  Material  and 
the  technical  bureaus  and  offices  to  develop  and 
implement  reliability  educational  programs  for 
military  personnel  assigned  to  technical,  con¬ 
tracting  and  quality  assurance  billets  and 
assigned  to  the  supervision  of  these  activities. 

2.  Develop  techniques  ,  methods,  and  knowledge 
in  the  field  of  human  reliability. 

3.  Participate  in  the  human  factors  analysis  for 
reliability  in  each  Navy  system  as  appropriate." 

5.  In  response  to  this  directive,  the  New  Developments  Branch  of 
the  Personnel  Research  Division  under  the  Chief  of  Naval  Personnel  initi¬ 
ated  this  research.  The  objectives  of  the  ORI  study  were  the  following: 

a.  Creation  of  methodologies  for  using  data  presently 
available  in  existing  Navy  failure  reporting  systems 
for  human  reliability  analysis. 

b.  Development  of  indirect  approaches  to  human  reli¬ 
ability  that  are  not  dependent  on  direct  data  on 
human  performance  but  can  be  derived  from  equip¬ 
ment  data . 


c.  Recommendations  for  further  research  efforts 
in  the  quantification  of  human  performance 
and  in  human  reliability  analysis. 

APPROACH 

6.  The  initial  efforts  of  the  study  were  to  establish  the  objectives 
for  the  work  and  to  establish  sound  definitions  of  human  reliability  termi¬ 
nology  which  could  serve  as  the  framework  for  the  remainder  of  the  study. 
These  definitions  are: 

a.  Human  Error  -  Any  action  of  the  human  element 
:c  z  system  that  is  inconsistent  with  a  predeter¬ 
mined  behavioral  pattern  established  in  the  sys¬ 
tem  specifications  and  in  the  resulting  system 
design. 

b.  System  Failure  -  Any  system  performance  that 
does  not  meet  the  requirements  established  by 
design  specifications  and  documentation.  Such 
failures  fall  into  two  categories:  (a)  total 
system  failure,  in  which  there  is  a  complete 
breakdown  in  performance,  and  (b)  system  de¬ 
gradation  failure,  in  which  the  system  .alls  to 
perform  at  its  specified  level  of  performance 
over  a  period  of  time. 

c.  Human  Reliability  -  Probability  that  human 
error  will  not  cause  a  system  failure  or  mal¬ 
function. 

The  scope  of  the  research  is  limited  to  human  reliability  as  related  to  the 
operational  and  maintenance  phases  of  the  weapon  systems  life  cycle. 

7.  Navy  failure  reportiny  systems  were  systematically  surveyed  to 
determine  the  relevance  of  their  data  to  human  reliability  analysis,  to  eval¬ 
uate  the  accuracy  of  such  data,  and  to  ascertain  other  pertinent  character¬ 
istics  such  as  range  of  hardware  covered  and  amount  of  data  available.  At 
the  completion  of  each  survey,  the  usefulness  of  the  data  for  human  reli¬ 
ability  analysis  was  objectively  evaluated.  The  following  systems  were 
surveyed: 
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a.  Electronic  Equipment  Failure/Replacement  Report 
(EFRR)  System 

b.  Casualty  Reporting  System  (CASREPT) 

c.  Fleet  Ballistic  Missile  Weapon  System  -  TFR 
Program 

d.  Surface  Missile  System  and  Air-Launch  Missile 
System  Performance  and  Failure  Reporting 

e.  Air-Launch  Guided  Missile  Systems  Performance 
Data  Reporting  Program 

f.  Surface  Missile  System  Equipment  Status  Log 

g.  Material  Maintenance  Management  (3-M)  System 

h.  Fleet  ASW  Data  Analysis  Program  (FADAP) 

8.  The  effort  dealing  with  the  indirect  approaches  consisted  of 
the  development  of  two  mathematical  techniques  neither  of  which  relies 
on  the  direct  reporting  of  human-initiated  failures  and  malfunctions . 

9 .  The  first  approach  is  termed  the  Elementary  Reliability  Unit 
Parameter  Technique  (ERUPT).  By  grouping  the  components  of  a  weapon 
system  into  Elementary  Reliability  Units,  ERUs  (the  lowest  levels  at 
which  maintenance  is  performed),  this  approach  provides  a  means  of  in¬ 
ferring  human  performance  parameters  from  available  equipment  reliability 
and  maintenance  data.  In  the  preliminary  stages  of  development  during 
this  study,  ORI  quantified  two  measures  of  human  performance  during 
maintenance  as  part  of  a  model  expressing  weapon  system  readiness. 

These  two  parameters  were: 

a.  The  probability  that  a  failure  Is  detected  and 
repaired  during  maintenance 

b.  The  probability  that  maintenance  does  not 
Induce  failure . 

The  model  consists  essentially  of  equations  and  computational  routines 
for  deriving  system  measures  of  effectiveness  from  failure  and  maintenance 
parameters. 

1C.  The  second  indirect  approach  uses  multivariate  correlation 

analysis  techniques  to  relate  certain  personnel  characteristics  of  indi¬ 
viduals  operating  and  maintaining  the  equipment  to  numbar  of  failures 
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and  equipment  repair  times .  Multivariate  correlation  analysis  is  a  well 
known  statistical  technique  used  to  measure  the  degree  or  the  importance 
of  the  relationship  between  a  dependent  variable  and  a  set  of  independent 
variables.  Data  obtained  on  two  equipments  from  the  3-M  system  were 
used  in  conjunction  with  personnel  characteristics  of  the  crew  associated 
with  those  equipments  during  selected  time  frames.  Although  the  analysis 
was  performed  only  on  a  pilot  basis  with  a  limited  set  of  data,  several 
significant  relationships  were  determined. 

FINDINGS,  CONCLUSIONS,  RECOMMENDATIONS 

11 .  During  the  first  phase  of  the  study  it  was  found  that  existing 
failure  reporting  systems  do  not  yield  meaningful  data  on  human-initiated 
malfunctions.  Most  systems  have  no  provisions  for  reporting  information 
on  human  performance  as  related  to  failures.  Many  of  the  reporting  systems 
surveyed  were  In  various  stages  of  being  phased  out  of  use.  In  most  cases, 
a  strong  reluctance  to  report  all  failures ,  particularly  human  errors ,  was 
noted.  What  Information  was  found  to  exist  on  human  errors  was  very  gen¬ 
eral  and  was  not  being  used  for  human  failure  analysis.  Indeed,  meaning¬ 
ful  analysis  of  the  reported  data  did  not  appear  feasible.  Some  provisions 
were  being  made,  however,  for  inclusion  of  human-initiated  failure  data 

in  the  3-M  system.  These  data  warrant  further  investigation  for  possible 
improvements  that  will  make  them  useful  in  human  reliability  analysis. 

12.  During  the  later  phases  of  the  study  the  feasibility  and  applica¬ 
bility  of  two  indirect  approaches  were  determined,  and  mathematical  models 
and  equations  were  developed  for  their  application.  A  pilot  test  on  a  limit¬ 
ed  data  base  was  also  conducted  during  the  development  of  each  techrtque. 
The  results  of  the  pilot  tests  revealed  that  both  techniques  are  potentially 
extremely  useful  tools  in  human  reliability  analysis,  and  that  they  could 
lead  to  significant  breakthroughs  in  the  quantification  of  human  performance. 

Recommendations  for  Further  Action 

a.  One  of  the  major  problems  associated  with 
failure  reporting  systems  is  their  use  as  an 
information  source  for  disciplinary  or  promo¬ 
tion-review  purposes  .  It  is  recommended  that 
BuPers  initiate  an  educational  program  to 
clarify  the  basic  purposes  of  these  reporting 
systems . 
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Another  problem  associated  with  reporting 
systems  is  the  design  of  the  forms  that  are 
used  to  report  failures.  It  is  recommended 
that  a  study  be  conducted  to  design  a  failure 
reporting  form  which  would  assure,  with  some 
degree  of  confidence,  the  correct  identifica¬ 
tion  of  the  causes  of  failures  . 

Based  on  the  feasibility  of  the  two  indirect 
approaches,  proven  in  this  study,  it  is 
recommended  that  they  be  further  developed 
by  extending  their  ap?1 'cation  to  a  larger 
data  base  and  to  other  classes  of  equipment. 
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I.  INTRODUCTION  AND  BACKGROUND 


GENERAL 

1 . 1  This  report  represents  the  results  of  a  research  study  on  human 
reliability  conducted  by  Operations  Research  Incorporated  (ORI)  under 
Contract  Nonr  4451(00)  for  the  New  Developments  Research  Branch, 
Personnel  Research  Division,  Bureau  of  Naval  Personnel. 

CONTRACT  BACKGROUND 


1.2  In  recent  years ,  the  Navy  has  become  more  aware  of  the  role  of 
the  human  element  in  system  effectiveness.  Increasing  emphasis  has 
been  placed  on  the  personnel  subsystem  in  new  system  development,  and 
the  need  for  new  tools,  and  techniques  in  human  reliability  analysis  and 
prediction  has  become  apparent.  The  recognition  of  this  nped  is  evi¬ 
denced  by  SECNAVINST  3900.36,  dated  27  January  1966.-^'*  The  purpose 
of  this  document  was  to  "establish  policy  for  the  guidance  of  efforts  to 
increase  the  reliability  of  naval  material  and  to  assign  responsibility 
for  its  achievement."  It  set  forth  the  objective  of  achieving  and  maintain¬ 
ing  "the  highest  level  of  reliability  in  naval  material  commensurate  with 
economic,  technological  and  logistics  constraints  in  context  with  the 
operational  requirements  for  the  material." 


Footnote  numbers  refer  to  correspondingly  numbered  items  in  Appen¬ 
dix  F . 
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1.3  It  also  assigned  to  the  Chief  of  Naval  Personnel  the  responsi¬ 
bility  to: 

"  1  •  Coordinate  with  the  Chief  of  Naval  Material 

and  the  technical  bureaus  and  offices  to  develop 
and  implement  reliability  educational  programs 
for  military  personnel  assigned  to  technical,  con¬ 
tracting  and  quality  assurance  billets  and  assigned 
to  the  supervision  of  these  activities. 

2.  Develop  techniques,  methods,  and  knowledge 
in  the  field  of  human  reliability. 

3 .  Participate  in  the  human  factors  analysis  for 
reliability  in  each  Navy  system  as  appropriate." 


1.4  This  research  program  was  then  initiated  in  response  to  SECNAVINST 
3900.  36  and  was  directed  toward  the  following  tasks: 

a.  Analyze  existing  failure  reporting  systems  to 
determine  the  extent  of  the  coverage  and  the 
availability  of  data  on  human-initiated  mal¬ 
functions  and  failures. 

b.  Determine  and  delineate  how  available  data  from 
these  systems,  as  they  presently  exist,  can  be 
used  in  support,  of  the  requirement  for  human 
reliability  analysis. 

c.  Recommend  and  define  further  research  to  refine 
and  improve  the  human  reliability  analysis  func¬ 
tion  and  th'’  systems  for  collecting  data  on  human- 
initiated  malfunctions. 

1.5  At  the  completion  of  tire  first  task,  it  became  apparent  that  the 
existing  failure  reporting  systems  did  not  yield  data  on  human  performance 
or,  more  directly,  on  human-initiated  malfunctions.  The  conclusion  led 
to  further  discussions  with  the  scientific  officer  and  resulted  in  the  re¬ 
direction  of  the  contra "t  effort.  ORI  proposed  to  explore  approaches  to 
human  reliability  which  would  not  depend  on  human  performance  data  from 
failure  reporting  systems  as  the  sole  or  primary  input.  The  contract  was  so 
redirected  and  the  research  effort  was  concentrated  on  the  investigation 
and  development  of  two  "indirect"  approaches  to  human  reliability. 
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ROLE  OF  HUMAN  RELIABILITY 


1. 6  Reliability  has  become  a  commonly  used  term  in  the  design, 
development,  production,  and  operation  of  weapon  systems.  It  is  generally 
accepted  as  one  of  the  most  significant  contributors  to  system  effective¬ 
ness  and  has  become  a  fundamental  characteristic  of  every  component, 
module,  subsystem,  or  total  system.  Until  recently,  the  concept  of  relia¬ 
bility  was  considered  only  as  it  applied  to  hardware  design  and  operation. 
However,  technological  advances  and  the  complexities  of  modern  weapon 
systems  are  imposing  severe  requirements  on  operating  and  maintenance 
personnel .  As  a  result,  the  importance  of  the  human  element  in  the 
determination  of  total  system  reliability  and,  subsequently,  total  system 
effectiveness  is  beginning  to  be  realized. 

1 .7  Over  the  past  decade,  the  contribution  of  the  personnel  subsystem 
to  system  reliability  and  effectiveness  and  the  degradation  in  system  per¬ 
formance  due  to  human  error  and  human-initiated  malfunctions  have  been 
subjects  of  numerous  studies  and  research  programs.  The  now-classic 
study  in  I960  by  Shapero, et  al.^*  gave  probably  the  first  major  impetus 

to  these  areas  of  investigation  by  showing  that  from  20  to  SA  percent  of 
all  system  malfunctions  in  nine  missile  systems  under  study  were  caused 
by  human  error  and/or  human-initiated  malfunctions.  Numerous  subsequent 
studies  by  Meister,  J^Cooper,^ Willis,  5/ and  others  have  substantiated 
the  fact  that  a  significant  percentage  of  system  unreliability  was  caused 
by  human  error. 

1 . 8  Concurrent  with  the  technological  advancements ,  the  techniques 
for  conducting  analysis  of  system  reliability  and  effectiveness  and  the 
prediction  of  hardware  reliability  have  also  advanced  considerably.  With 
more  extensive  usage  of  the  computer  and  automatic  data  processing  (ADP), 
complex  models  are  being  applied  through  the  use  of  volumes  of  data  from 
laboratory  and  field  operations.  The  degree  of  sophistication  achieved  in 
analytic  techniques  has  created  a  science  of  systems  analysis  and  replaced 
intuition  with  mathematical  calculations  . 

1 .9  The  criticality  of  the  human  element  in  system  effectiveness  and 
the  development  of  more  sophisticated  system  analysis  techniques  places  an 
added  burden  on  human  factors  specialists  as  they  try  to  assume  their  role 
and  perform  their  functions  In  new  weapon  system  development.  In  order  to 
participate  meaningfully  in  new  system  development  and  to  influence  equip¬ 
ment  design,  they  must  deal  in  terms  that  are  commensurate  with  the  tools 
and  information  available  to  systems  and  design  engineers.  The  obvious 
conclusion  is  the  often-stated  requirements  for  the  quantification  of  human 
performance. 
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1 ,  iO  Human  factors  or  human  engineering  does  not  yet  enjoy  the  wide¬ 
spread  acceptance  in  system  engineering  circles  that  it  deserves.  In  spite 
of  the  results  of  the  previously  cited  studies,  some  people  fail  to  recognize 
the  criticality  of  the  human  element  in  system  effectiveness.  These  same 
people  will  continually  look  for  more  advanced  techniques  in  hardware  design 
with  a  total  disregard  for  the  human  component.  Others,  on  the  other  hand, 
arbitrarily  and  indiscriminately  blame  most  problems  on  lack  of  training  or 
failure  of  supporting  personnel  to  follow  procedures.  To  convince  systems 
systems  analysts  of  the  importance  of  the  human  element  in  systems 
analysis  and  to  determine  the  contribution  of  the  various  characteristics 
of  the  personnel  subsystem  to  human-initiated  failures,  there  is  a  dire 
need  for  factual,  quantified  data  on  human  performance. 

1.11  It  is  to  these  problems  that  this  research  program  on  human 
reliability  was  directed.  The  heart  of  reliability  analysis  and  prediction 
is  the  availability  of  extensive  statistical  or  historical  data  derived  from 
failure  reporting  of  weapon  system  malfunctions  while  under  test  or  during 
operation  and  maintenance.  Through  the  analysis  of  failure  reports,  the 
malfunctions  of  a  component,  subsystem,  or  system  can  be  pinpointed  and 
causes  determined.  As  it  applies  to  equipment,  this  procedure  has  been 
initiated  and  has  been  in  operation  for  some  time  in  all  branches  of  the 
Navy.  Similarly,  there  is  a  definite  need  for  reporting  and  analyzing 
human-initiated  malfunctions  and  for  determining  their  causes. 

ORGANIZATION  OF  REPORT 

1 .12  Section  II  discusses  the  need  for  common  terminology  and 
definitions  in  the  human  reliability  field.  Some  basic  definitions  which 
set  the  framework  for  this  study  and  a  detailed  breakdown  and  identifi¬ 
cation  of  the  various  types  of  human  error  are  also  provided. 

1.13  Section  II  also  describes  the  broad  scope  of  the  human  relia¬ 
bility  study  as  it  applies  to  the  total  life  cycle  of  a  system  from  its  con¬ 
ception  to  its  last  day  of  operational  usefulness.  It  addresses  the  problem 
of  dealing  with  human  reliability  in  such  a  broad  context  and  calls  for  the 
limitation  in  scope  to  particular  phases  in  the  life  cycle  of  weapon  systems. 

1.14  Section  III  relates  the  results  of  the  first  phase  of  the  study 
which  dealt  with  the  investigation  and  analysis  of  existing  failure 
reporting  systems  in  relation  to  their  treatment  of  human  performance  and 
human-initiated  malfunctions.  The  section  discusses  the  most  significant 
problems  associated  with  the  reporting  of  failures  caused  by  human  errors 
and  offers  recommendations  regarding  basic  improvements  in  failure  reporting 
systems . 
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1.15  Section  IV  discusses  the  need  for  indirect  approaches  to  human 
reliability  which  would  not  be  dependent  on  the  direct  reporting  of  human- 
initiated  malfunctions.  Two  approaches  developed  by  ORI  are  described 
and  the  results  of  pilot  applications  of  these  techniques  are  provided. 

1.16  Section  V  summarizes  the  study  res  'lts  and  the  conclusions 
derived  from  their  analyses.  A  set  of  recommendations  for  specific  actions 
to  be  instituted  by  the  Navy  and  for  future  research  studies  is  also 
provided . 

1. 17  Appendices  A  through  D  contain  mathematical  derivations,  pilot  test 
results,  and  computer  programs  for  one  of  the  two  indirect  approaches  to 
human  reliability.  Appendix  E  describes  the  methodology  used  to  evaluate 
hypotheses.  Appendix  F,  the  bibliography,  lists  (a)  works  referenced  in 
the  report  and  (b)  works  used  for  background  information. 
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II .  DEFINITION  OF  T  ERMS 


INTRODUCTION 

2.  1  One  of  the  prerequisites  for  conducting  research  in  any  dis¬ 
cipline  is  a  set  of  well-defined  terms  or  parameters  that  are  widely  used 
by  researchers  in  the  field.  For  example,  in  the  "hardware  world"  such 
terms  as  reliability,  MTBF,  performance  measurement,  etc.,  have  been 
used  for  a  number  of  years  and  have  acquired  universal  meaning.  One 
continuing  problem  tnat  has  plagued  researchers  in  the  field  of  human  re¬ 
liability  is  the  lack  of  a  well-established  glossary  of  terms  to  provide 
a  common  base  for  study  and  analysis  conducted  in  this  field. 

2.2  A  report  by  the  Department  of  Psychology  of  t  .e  University  of  Southern 
California^/  cites  a  good  example  of  this  lack  of  common  terminology  by 
quoting  some  definitions  of  maintainability  from  the  EIA  Task  Group's  Guide: 

"Maintainability  is  a  quality  of  the  combined  features 
and  characteristics  of  equipment  design  which  permits 
or  enhances  the  accomplishment  of  maintenance  by 
personnel  of  average  skills,  under  the  natural  and 
environmental  conditions  in  which  it  will  appear." 

"Maintainability  is  the  assurance  that  specified  main¬ 
tenance  procedures  v/ill  be  completed  in  a  given 
environment  within  a  satisfactory  time." 

"Maintainability  is  the  ease  with  which  a  device 
can  be  kept  operating." 
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" Maintainability  can  .  .  .be  defined.  .  .as  the 
probability  that  the  system  can  be  returned  to 
service  in  a  given  period  of  time.  Maintain¬ 
ability  as  thus  def-ued  can  be  measured  by  the 
system  mean-downtime. " 

"Maintainability  is  a  measure  of  the  speed  and 
ease  with  which  preventive  maintenance  can 
be  performed  or  equipment  malfunctions  diagnosed 
and  corrected. " 

2.3  On  the  other  hand,  the  Reliability  and  Maintainability  Training 
Handbook  prepared  by  General  Dynamics  Astronautics  7/  offers  the 
following  definition: 

"Maintainability  is  the  speed  or  economy  with  which  a  system 
or  component  can  be  kept  in,  and/or  restored  to, 
full  performance  capability.  A  principally-used  measure 
is  the  average  number  of  failures  lestored  per  hour 
of  Corrective  Maintenance  time  which  is  the  reciprocal 
of  MTTR.  Another  is  the  fraction  of  attempts  wherein 
restoration  is  completed  in  a  specified  time,  or  the 
probability  that  it  will  be  completed  in  that  time. 

Another  is  the  functional  time  obtained  per  dollar 
cost  of  preventive  and  corrective  maintenance." 

2.4  Human  reliability  in  itself  has  not  been  defined  adequately,  and 
researchers  have  attached  different  meanings  to  the  whole  concept.  Meistec,^ 
as  an  example,  defines  human  reliability  as  "the  probability  that  a  job 
operation  will  be  successfully  performed  by  personnel  at  any  required  stage 

in  system  operation  within  a  criterion  time  period. " 

2.5  Rabldeau  offers  the  following  definition:  "Personnel  Subsystem 
reliability  is  a  function  of  the  frequency  of  occurrence  of  human  error  in  the 
execution  of  required  system  functions,  insofar  as  such  error  affects  the 
system's  outputs  and  component  conditions.  " 

2.6  Captain  Majesty  of  the  Air  Force  Ballistic  Systems  Division 
attempted  to  express  human  reliability  in  terms  of  a  figure  of  merit  and 
"consequently,  human  reliability  can  be  compared  to  hardware  reliability 
.  ,  ,  .Human  performance,  like  hardware  performance,  must  satisfy  the 
performance  requirements  of  the  system.  " 

2.7  Similarly,  most  terms  used  in  the  field  of  human  reliability  or 
the  quantification  of  human  performance  lack  common  definition. 
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DEFINITIONS 


2.8  To  provide  a  framework  for  this  study  and  to  attempt  to  reduce 
some  of  this  confusion,  the  following  three  terms  are  defined  as  they  are 
used  in  this  report: 

a.  Human  error.  Any  action  of  the  human  element  of 
a  system  that  is  inconsistent  with  a  predetermined 
behaviorial  pattern  established  in  the  system  spec¬ 
ifications  and  in  the  resulting  system  design. 

b.  System  failure  or  malfunction.  Any  system  performance 
that  does  not  meet  the  requirements  established  by  de¬ 
sign  specifications  and  documentation.  This  definition 
further  breaks  down  into  two  types  of  failures:  total  system 
failure,  in  which  there  is  a  complete  breakdown  in 
performance,  and  system  degradation  failure,  in  which 

the  system  fails  to  perform  up  to  a  specified  level  of 
performance  over  a  period  of  time. 

c.  Human  reliability.  Probability  that  human  error  will 
not  cause  a  system  failure  or  malfunction. 

Types  of  Human  Error 

2.9  Note  that  human  errors  that  do  not  result  in  system  failure  are 
excluded  from  human  reliability  considerations  in  this  report.  As  discussed 
by  Meist^r  in  1964,  — '  only  errors  that  affect  the  system  performance  have 
any  meaning  to  the  system  analyst.  As  an  example,  the  failure  to  perform 
an  act  which  is  discovered  in  time  and  is  subsequently  performed  would  he 
considered  a  human  error,  as  previously  defined;  but,  since  it  has  no  bearing 
on  system  performance,  it  would  not  be  considered  in  any  human  reliability 
analyses.  Further,  because  of  the  limitations  imposed  by  data  retrieval, 
discussed  later  in  this  report,  obtaining  information  on  this  type  of  human 
error  on  a  continuous  basis  is  nearly  impossible. 

2.10  The  lack  of  commonality  in  the  attempt  to  classify  the  types  of 
human  error  is  also  evident  in  the  works  of  researchers  in  this  field.  The 
following  are  examples  of  some  of  these  classifications. 

"Terminal  error  -  all  deviations  from  procedures  which 

always  result  in  failure  of  the  operation. 

Risk  error  -  omission  of  prescribed  precautionary  measures. 

Residual  error  -  all  other  deviations  from  procedure."  — / 

"Design  error  -  failures  resulting  from  inadequate  design. 

Fabrication  error  -  failures  resulting  from  poor  workman- 

sh,p  in  the  factory. 


'Operating  error  -  failure  resulting  from  personnel  operating 
the  equipment  incorrectly  in  the  field.  There  are  several 
subcategories: 

1 .  failure  to  follow  procedures 

2.  use  of  incorrect  procedures  or  lack  of 
correct  procedures 

3.  use  of  improper  tools  or  lack  of  correct 
tools 

4.  motivational  error 

'Maintenance  error  -  failure  resulting  from  incorrect 
installation  or  repair  of  equipment  in  the  field. 

1 .  repair  error 

2.  installation  error 

3 .  calibration  error 


‘Contributory  error  —  failure  resulting  partially  from 
mechanical-electrical-electronic  factors  and  partially 
from  human  error . 


"Performance  of  a  required  action  incorrectly. 
Failure  to  perform  the  required  action. 
Performance  of  a  required  action  out  of  sequence , 
Performance  of  a  non-required  action,  "iiv 


"Errors  of  omission 

1 .  errors  of  memory 

2 .  errors  of  attention 


"  Errors  of  commission 


1  .  errors  of  identification 

2.  errors  of  interpretation 

3.  errors  of  operation."!^/ 

2.11  In  this  report,  the  types  of  human  error  are  broken  down  into  two 
major  categories,  predictable  human  errors  and  random  human  errors. 

2.12  Predictable  human  error  refers  to  those  occasions  in  which  a  causal 
relationship  can  be  established  between  the  inconsistent  behavior  (with 
respect  to  system  specifications)  and  some  external  influence.  There  is  a 
high  degree  of  probability  associated  with  the  recurrence  of  these  errors 
under  Identical  or  even  similar  circumstances.  The  number  of  this  type  of 
error  can  be  reduced  by  modifying  the  external  influences  through  the 
redesign  of  either  the  hardware  or  personnel  subsystems  or  by  the  change 

of  system  specifications. 


10 


2.  13  With  respect  to  random  human  errors,  it  is  recognized  that  the 
human  element  represents  the  most  unpredictable  and  complex  component 
of  any  system, since  its  psychophysical  makeup  creates  a  tendency  toward 
random,  nonpredictable  behavioral  patterns  .  Some  of  this  irregular  be¬ 
havior  cannot  be  attributed  to  any  particular  cause,  and  other  causes, 
even  though  identifiable,  will  not  logically  be  considered  as  criteria  for 
a  design  change.  Further,  there  is  a  very  low  degree  of  probability 
*hat  under  similar  or  even  identical  circumstances  such  errors  will  recur. 
This  category  of  errors  is  termed  random  error. 

2.14  Figure  1  illustrates  a  representative  breakdown  of  various  types 
of  human  errors  which  may  be  structured  in  the  form  of  a  failure  or  error 
analysis  tree.  The  cause  for  any  system  or  subsystem  failure  is  first 
categorized  as  either  an  equipment -initiated  malfunction  or  a  human- 
initiated  malfunction.  If  the  human  element  is  assumed  to  be  the  causal 
agent,  the  human  error  or  failure  can  be  further  reduced  to  the  two  elements 
of  predictable  and  random  human  errors.  If  it  is  deemed  that  the  failure  was 
caused  by  a  random  error,  a  component  of  human  reliability  has  been  iden¬ 
tified  and  the  analysis  is  therefore  completed. 

2.15  The  design  element  of  predictable  error  can  be  further  broken  down 
into  equipment  design  and  procedural  problems.  Equipment  design  includes 
what  is  normally  considered  a  human  engineering  problem,  in  which  the 
hardware  design  has  an  apparent  deficiency  from  a  human  factors  standpoint. 
This  type  of  error  should  be  reverted  back  to  equipment  failure  and  be  in¬ 
cluded  in  equipment  reliability  considerations.  Thus,  it  is  either  corrected 
by  equipment  redesign  or,  because  of  cost  or  other  considerations,  retained 
as  a  inherent  reliability  characteristic  and  figured  into  subsequent  MTBF 
considerations . 

2.16  Under  procedural  problems,  two  subcategories  can  be  considered, 
those  that  involve  equipment  (tool)  availability  and  those  that  involve  the 
availability  of  proper  procedures,  which,  in  the  negative  sense,  could  mean 
the  complete  lack  of  tools  or  procedures  or  the  availability  of  improper 
ones.  Here  again  the  analysis  is  completed  since,  essentially,  the  human 
element  has  been  taken  out  of  the  causal  relationship  and  the  problem  is 
reverted  to  a  system  redesign  in  the  procedural  or  support  equipment  area. 

2.17  It  is  pointed  out  that  these  types  of  errors  do  not  include 

the  selection  of  improper  equipment  (tools)  or  the  selection  of  improper  pro¬ 
cedures,  since  both  cases  can  be  considered  as  either  random  error  or 
a  personnel  characteristic  that  is  an  element  of  predictable  error,  as  de¬ 
fined  in  the  ensuing  paragraph. 

2.18  The  personnel  characteristics  aspect  of  predictable  human  errors 
entails  all  those  errors  in  the  latter  category  that  are  directly  attributable 


to  the  operator  or  to  the  composition  of  the  personnel  subsystem .  Even  in 
this  category,  a  further  breakdown  yields  a  component  that  results  from 
influences  external  to  the  personnel  subsystem.  This  subcategory  is 
termed  "environmental  conditions"  and  covers  those  human  errors  that 
are  caused  by  external  factors  such  as  weather  conditions,  combat  con¬ 
ditions,  or  sea  states. 

j 

2.19  The  number  of  occurrences  of  such  errors  can  be  reduced  by  a 
system  redesign  in  which  such  things  as  relocation  of  certain  equipment 
within  the  overall  system  configuration  are  considered,  or  by  a  speci¬ 
fication  change  where,  for  example,  using  the  system  under  certain  con¬ 
ditions  is  deemed  unfeasible. 

2.20  The  remaining  component  of  personnel  characteristics  is  identi¬ 
fied  as  personnel  composition.  Here,  we  finally  arrive  at  that  aspect  of 
predictable  human  error  which  is  directly  attributable  to  the  human  element 
and  results  from  such  contributing  factors  as  training,  staffing,  selection, 
and  motivation. 

2.21  A  detailed  discussion  of  motivational  errors  in  psychological  terms 
is  outside  the  scope  of  this  report  except  for  the  random  vs  predictable 
error  classifications  that  can  also  be  applied  to  this  area.  There  are 
"interpersonnel"  motivational  errors  which  result  from  insufficiency  or 
lack  of  motivation  of  an  entire  group,  and  they  should  be  the  subject  of 
system  redesign  consideration.  "Intra personnel"  motivational  errors 
involve  the  psychological  problems  of  individuals  owing  to  special  cir¬ 
cumstances  and,  as  such,  are  considered  random  errors  and  would  not 

be  considered  in  system  design.  The  other  three  components  of  personnel 
composition  meed  no  further  elaboration. 

2.22  It  is  significant  to  note  that  the  last  four  factors  (traditional  ele¬ 
ments  of  manpower  management)  represent  only  one  of  the  final  links  in 
this  analysis  tree,  even  after  a  human  error  has  been  established  as  the 
cause  for  system  failure  or  malfunction.  Yet,  when  conducting  interviews 
in  the  field,  there  is  a  tendency  to  attribute  all  breakdowns  in  the  personnel 
subsystem  to  this  one  component  and  to  the  training  factor  in  particular. 
However,  until  we  can  quantify  human  performance  and  determine  the 
relative  magnitudes  of  the  various  components  at  different  levels  of  the 
analysis  tree,  it  is  difficult  to  disprove  this  misconception.  If  human 
reliability  analysis  techniques  are  going  to  be  developed  in  a  quantifiable 
form  commensurate  with  system  reliability  techniques,  data  will  have  to  be 
made  available  to  support  analysis  as  outlined  in  the  foregoing  discussion. 

LIMITATIONS  IN  SCOPE 

2.23  The  concepts  of  human  performance  and  human  reliability  have 
an  application  In  all  stages  of  the  life  cycle  of  a  system,  from  its  very 
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beginning  In  the  conceptual  phase  to  the  last  day  of  Its  operational  use¬ 
fulness.  Further,  all  system  failures  and  malfunctions  (except  certain 
laboratory  tests)  can  eventually  be  traced  jpack  to  some  form  of  human 
error,  whether  it  occurred  on  the  drawing  board,  in  fabrication,  in  test¬ 
ing,  in  operation,  or  in  maintenance.  Along  with  a  lack  of  definitions 
and  uniform  terminology,  the  consideration  of  the  term  "human  reliability" 
in  such  a  broad  context  is  one  of  the  factors  most  detrimental  to  perform¬ 
ing  research  in  this  field.  By  attempts  to  analyze  human  reliability 
during  the  total  life  cycle  of  the  system,  all  efforts  are  diffused  to  the 
point  that  it  is  extremely  difficult,  if  not  impossible,  to  achieve  a  signi¬ 
ficant  breakthrough.  It  therefore  becomes  imperative  to  limit  the  scope 
of  our  research  to  certain  specific  phases  of  the  system's  life  cycle. 

2.24  It  is  considered  that  human  reliability  during  design  and  produc¬ 
tion  phases  are  subjects  of  a  separate  area  of  analysis.  In  no  way  does 
this  distinction  minimize  the  criticality  of  human  error  in  these  two  phases, 
and  there  is  a  definite  need  for  additional  research  in  those  areas.  They 
are  not,  however,  a  part  of  human  reliability  as  defined  in  this  report. 
Rather,  they  are  a  part  of  equipment  reliability  that  is  Inherent  to  the  sys¬ 
tem  when  it  is  assimilated  into  the  Navy  inventory  at  the  beginning  of  its 
operational  phase.  By  definition,  then,  this  report  deals  with  human  re¬ 
liability  during  the  operational  phase  (which  includes  operation  and  main¬ 
tenance)  of  the  system  life  cycle. 

2.25  Failure  reporting  systems  were  scrutinized  for  data  in  both  opera¬ 
tion  and  maintenance  areas  of  the  system  life  cycle.  Emphasis  was  placed 
on  the  maintenance  area  in  the  indirect  approaches  because  maintenance 
"success"  seems  to  depend  more  on  the  human  element  than  on  the  opera¬ 
tional  environment.  Also  maintenance  criteria  were  more  easily  quantifiable. 
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III.  HUMAN-INITIATED  MALFUNCTION  DATA 


GENERAL 

3. 1  Considerable  effort  has  been  expended  by  researchers  in  an  attempt  to 
quantify  human  performance.  Numerous  models  and  equations  have  been  devel¬ 
oped  and  many  of  them  appear  to  be  quite  useful  tools  for  predicting  and/or 
measuring  human  reliability.  Most  researchers  agree,  however,  that  none 

of  these  models  can  be  applied  with  any  predictable  degree  of  success 
until  they  can  be  validated  through  actual  data  obtained  from  the  field. 

3.2  Here,  again,  equipment  reliability  prediction  and  analysis  have 
a  significant  jump  on  similar  efforts  in  human  reliability.  As  Melster 
points  out,  the  prediction  of  equipment  reliability  is  based  on  historical 
record  or  performance  data  and  the  logic  of  assumed  similarity  between 
equipments.  Throughout  the  years,  a  wealth  of  data  has  been  collected  on 
equipment  performance  which  now  forms  the  basis  for  reliability  analyses. 

Of  course,  collecting  data  on  equipment  performance  is  considerably 
simpler  than  obtaining  data  on  human  performance.  One  can  test  equip¬ 
ment  under  controlled  and  simulated  conditions  in  the  laboratory  with 
reasonable  assurance  that  the  same  equipment  will  perform  in  the  same 
manner  under  similar  conditions  in  the  field.  Unfortunately,  the  human 
element  is  far  more  complex.  One  cannot  test  an  individual  in  the 
laboratory  and  then  predict  with  any  degree  of  assurance  that  he  will  per¬ 
form  in  the  same  manner  under  actual  field  conditions.  There  are  so  many 
variables  that  can  and  do  affect  human  behavior  and,  in  turn,  performance, 
that  it  is  virtually  impossible  to  duplicate  or  simulate  them  under  a  con¬ 
trolled  test  environment.  And,  even  if  it  were  possible  to  test  an  indi¬ 
vidual  in  the  laboratory  and  then  predict  his  behavior  in  the  field,  there 

is  little  assurance  that  another  individual  would  behave  in  a  similar 
manner. 
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3.3  These  problems  do  not  indicate,  as  is  sometimes  felt,  that  it  is 
useless  to  try  to  obtain  data;  rather,  they  point  to  the  fact  that  we  need 
even  more  data  to  overcome  the  problems  to  perform  human  reliability 
analyses  than  we  need  to  perform  equipment  reliability  analyses. 

HUMAN-INITIATED  MALFUNCTION  REPORTING 

3.4  At  the  onset  of  this  research  study,  the  decision  was  made  that, 
rather  than  try  to  develop  a  new  model  for  predicting  human  reliability, 

an  investigation  should  be  made  as  to  what  data  are  presently  available 
in  the  Navy  that  can  be  used  for  human  reliability  analysis .  To  accomplish  this 
investigation ,  an  intensive  review  wa  s  conducted  of  the  failure  reporting  systems 
presently  used  by  the  Navy.  The  investigation  revealed  that  these  systems 
are  almost  completely  void  of  any  information  dealing  with  the  human  ele¬ 
ment.  The  first  and  most  obvious  item  looked  for  was  some  direct  means  of 
distinguishing  between  equipment  failures  and  human-initiated  malfunctions. 

None  of  the  systems  directly  yields  data  that  will  permit  this  type  of  distinc¬ 
tion.  In  some  cases,  the  list  of  codes  used  to  identify  the  type  and/or 
cause  of  failure  does  not  even  include  any  data  that  could  be  associated 
with  the  human  element. 

3.5  The  following  paragraphs  describe  the  various  systems  investigated 
during  this  study.  Copies  of  forms  and  printouts  used  by  these  systems 
were  obtained  and  numerous  individuals  were  interviewed  who  are  presently 
or  were  then  actively  associated  with  these  systems. 

Electronic  Equipment  Failure/Replacement  Report  (EFRR)  System 

3.6  This  reporting  system  was  initiated  in  May  1961  by  the  Bureau  of 
Ships  [presently  Naval  Ships  System  Command  (NAVSHIPSYSCOM)] .  The 
system  requires  that  a  report  be  completed  (BuShips  10550-1)  for  every  re¬ 
pair  action  that  involves  the  failure  and/or  replacement  of  electronic, 
electrical,  or  mechanical  parts,  units,  or  assemblies  in  equipments  speci¬ 
fied  in  the  Electronic  Information  Bulletin  (NAVSHIPS  900.002A).  This  requirement 
applies  to  all  facilities  using  or  repairing  NAVSHIPSYSCOM  electronic  equipment. 

3.7  With  the  advent  of  the  Material  and  Maintenance  Management  (3-M) 
system,  the  EFRR  has  been  completely  phased  out  as  it  applies  to  operation 

and  maintenance  aboard  ships.  The  EFRR  system  is  still  operational  in  shore  fa^ 
cilities,  and  there  is  no  evidence  of  any  official  document  authorizing  or  di¬ 
recting  the  complete  phase-out  of  the  system. 

3.8  The  Electronics  Maintenance  Engineering  Center  (EMEC)  is  re¬ 
sponsible  for  the  operation  of  the  system.  Data  gathered  through 
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EFRR  are  available  at  this  facility.  Under  the  new  organization,  EMEC 
is  presently  identified  as  Norfolk  Division,  Naval  Ship  Engineering 
Center  (NORDZV,  NAVSEC).  A  computer  program  is  presently  being  developed 
by  this  agency  which  will  accept  both  EFRR  and  3-M  data  to  provide 
continuity  in  existing  reports  and  ongoing  analyses. 

3.9  The  report  completed  by  the  technician  or  engineer  making  the 
replacement  or  discovering  the  failure  is  forwarded  to  Code  6?9C,  BuShips 
(now  NAVSHIPS).  Prior  to  1965,  analysis  of  the  data  was  performed  by 
private  contractors  and  the  results  and  forms  were  sent  to  EMEC,  Norfolk. 
Since  1965,  EMEC  has  been  performing  all  analyses  and  data  processing 
of  the  information  made  available  by  the  system. 

3.10  Figure  2  is  a  copy  of  the  report  form  used  by  the  system. 

The  form  does  not  identify  the  individual  who  operates  or  maintains  the 
equipment,  but  rather,  the  individual  who  discovers  the  failure.  The  two 
may  or  may  not  be  the  same  person  (Item  2).  Items  4  and  5  can  yield 
information  from  which  equipment  down-time  can  be  calculated.  Items  18 
and  20  provide  type-of -failure  and  cause-of-faiiure  codes,  respectively. 
Tables  1  and  2  set  forth  the  codes  used  in  these  two  categories.  No 
codes  in  either  category  are  applicable  to  human-initiated  malfunctions. 

A  further  problem  associated  with  the  system  is  that  reports  are 

often  not  completed  for  failures  which  did  not  require  replacement  of  parts 

and  were  repaired  immediately. 

Casualty  Reporting  System  (CASREPT) 

3.11  Authority  and  details  of  CASREPT  are  found  in  "Operations 
Report"  (NWIP  10-1  (A)  Article  510,  classified  CONFIDENTIAL  and 
NAVMATINST  4000.23,  dated  25  Aug  66). 

3.12  A  casualty  is  defined  as  an  Impairment  of  any  resource,  including 
personnel,  which  does  not  permit  full  combat  readiness  of  the  naval  element. 
Inputs  to  CASREPT  of  completed  questionnaires  are  received  in  NAVSUPSYS- 
COMHQ.  The  items  answered  on  these  questionnaires  for  each  casualty 

are  shown  in  Table  3  . 

3.13  All  naval  elements,  such  as  ships  and  shore  stations,  are  respon¬ 
sible  for  reporting  casualties  that  occur.  The  purpose  of  CASREPT  is  to 
keep  CNO  informed  on  the  current  combat  readiness  of  all  naval  elements. 
The  codes  used  to  identify  combat  readiness  are  shown  in  Table  4.  This 
system  has  been  modified  to  coordinate  it  with  the  3-M  system. 

3. 14  The  most  relevant  input  item  to  the  CASREFT  report  is  the  cause  of 
the  casualty.  Code  0411  assigns  one  of  the  following  base  cause  codes  to 
the  reason  supplied  by  the  Commander  in  charge  of  the  naval  element: 

a.  Material  failure 

b.  Design  failure 
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TABLE  2 

CAUSE- OF-FAILURE  CODES 


Code 

Cause 

1 

Prime  power  surge 

2 

Handling  damage 

3 

Shipping  damage 

4 

Shock  or  vibration 

5 

Extreme  cold  ambient  temperature 

6 

Extreme  hot  ambient  temperature 

7 

Result  of  other  part  failure 

8 

Deterioration  in  use 

9 

Deterioration 

0 

Other,  explain  under  "remarks" 

19 


f'OCCLANT  or  FOCCPAC  Repair  Activities 


c.  .  Personnel  error 

d .  Storm 

e.  Collision 
t.  Grounding 

g.  Fire 

h.  Explosion 

i.  Unknown 

3.15  Outputs  may  be  summarised  by  any  of  these  base  codes  or  by 
categories  of  the  other  input  items.  Discussions  with  personnel  associated 
with  the  system  revealed  that  the  percentage  of  times  Item  c  is  indicated  as 
the  base  cause  varies  from  1  to  10  percent  of  all  codes  used  depending  on 
the  equipment  in  question.  In  all  cases,  the  percentage  is  low  and  is  not 
believed  to  reflect  actual  conditions. 

3.16  Casualties  are  reported  to  Code  0411  when  they  are  considered 
"significant"  by  the  Commander  in  charge.  Thus,  because  of  the  strong  naval 
tradition  to  accomplish  the  job  regardless  of  adversities,  it  is  questionable 
whether  all  significant  casualties  are  promptly  reported  to  Code  0411. 
Commanders  in  charge  realize  that  an  abnormally  high  number  of  reported 
casualties  may  affect  promotions. 

Fleet  Ballistic  Missile  (FBM)  Weapon  System  -  Trouble  and  Failure 
Report  (TFR)  Program* 

3.17  The  FBM  Weapon  System  TFR  Program  has  been  established  to 
communicate  troubles  and  failures  and  associated  corrective  actions  among 
FBM  activities  under  the  cognizance  of  the  Special  Projects  Office  (SPO). 

The  system  is  also  used  by  FBM  Weapon  System  contractors  as  a  basis  for 
corrective  action  in  manufacturing  processes  end  quality  control,  for  system 
modifications  and  redesign,  and  for  correcting  document  errors. 

3. 18  The  TFR  Program  consists  of  four  major  elements: 

a.  Reporting  of  FBM  troubles  and  failures 

b.  Reporting  of  operating  time  on  FBM  equipment 

c.  Analysis,  evaluation,  and  corrective  action 

d.  Information  feedback  and  Fleet  liaison. 


*  Most  of  the  Information  presented  here  regarding  the  TFR  Program  was 
taken  from  SP  Instruction  3100.  lB.Ai/ 
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The  vehicle  for  reporting  troubles  and  failures  is  SP  Form  3100. 1A  (TFR), 
shown  as  Figure  3 . 

3.19  The  Fleet  Missile  System  Analysis  and  Evaluation  Group  (FMSAEG), 
under  the  direction  of  SPO,  is  responsible  for  the  administration  of  the  FBM 
Weapon  System  TFR  Program  and  the  performance  of  certain  operations  within 
the  program.  FMSAEG  maintains  and  operates  a  center  for  receiving  all  TFRs, 
Elapsed  Time  Meter  Records  (ETMRs) ,  and  other  records  completed  by  all 
participating  activities  and,  in  turn,  provides  copies  of  these  records  to 
appropriate  activities.  This  group  continuously  analyzes  all  data  resulting 
from  TFRs  to  identify  problem  areas,  failure  trends,  or  system  quality  or 
reliability  degradation.  The  group  also  initiates  TFR  Corrective  Action  Reports 
(CARs),  shown  in  Figure  4,  as  appropriate.  FMSAEG  maintains  a  computer  file  of 
TFR  data  generated  which  provides  a  central  data  bank  of  historical  failure 
information . 

3.20  The  TEMRs  have  been  developed  for  the  reporting  of  operating  times 
on  FBM  equipments  so  that  the  malfunctions  which  have  been  separately  re¬ 
ported  on  TFRs  can  be  correlated  with  the  operating  time  of  the  pertinent 
equipment.  A  copy  of  the  ETMR  is  presented  as  Figure  5. 

3.21  Corrective  action  reporting  in  the  TFR  Program  is  one  of  the  most 
significant  features  of  the  system  because  it  provides  a  closed  loop  for  the 
disposition  of  system  failures  and  malfunctions.  All  contractors  in  the  FBM 
Program  participate  in  the  TFR/CAR  system.  This  system  provides  strong 
assurance  that  the  malfunctions  will  be  well  defined  and  that  the  cause  of  the 
failure  will  be  determined,  ihus  initiating  proper  corrective  action.  FMSAEG 
prepares  and  distributes  monthly  TFR  Summary  Reports,  the  basic  purpose  of 
which  is  to  provide  timely  feedback  of  corrective  action  information  to  program 
participants.  Figure  6  is  a  flow  chart  which  portrays  the  action  and  the 
information  flow  of  the  TFR  Program. 

3.22  The  analysis  and/or  editing  of  TFRs  is  accomplished  with  the 

TFR  Worksheet  (Figure  7) .  This  worksheet  is  designed  to  permit  the  trans¬ 
cription  of  the  information  contained  on  the  TFR  to  a  format  that  enables  the 
key-punching  of  the  desired  information  onto  cards  and  transfer  to  tape. 
Information  storage,  processing,  and  retrieval  is  accomplished  through  a 
computer  program  called  Variable  Information  Processing.  Detailed  instructions 
for  completing  the  worksheet  provide  lists  of  words  and  phrases  of  common 
terminology  to  be  used  when  completing  Blocks  26  through  30.  Two  of  these 
lists,  "Types  of  Trouble  or  Failure"  and  "Description  of  Trouble  or  Failure," 
which  are  of  significance  in  the  content  of  this  report  are  shown  in  Tables  5 
and  6,  respectively. 

3.23  As  evidenced  by  the  foregoing  discussion,  the  TFR  program  is 
primarily  geared  to  equipment  and  total  system  considerations.  Little  or  no 
provision  is  made  for  dealing  with  the  human  element.  Admittedly,  the 
description  of  the  type  and  cause  of  a  failure  is  accomplished  through 
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TABLE  5 

TFR  TYPES  AND  DESCRIPTIONS 


Assembly 


Documentation 
Expended  normally 


Installation 


Installation  spare 


Overhaul 


Overhaul  spare 

PMMP-627  -A 
(for  example) 

Recall 


Receiving 


Recertification 


Repair 


Repair  spare 


Routine 


The  failure  occurred  or  was  discovered  during  assembly  operations  and 
testing,  such  as  ml;  lie  assembly  at  PMFA/MAB, 

The  TFR  reports  a  document  Inadequacy. 

The  component,  such  as  a  missile  battery,  performed  satisfactorily  and 
was  expended  in  the  process,  in  this  case,  the  TFR  Is  used  as  a  shipping 
document . 

The  failure  occurred  or  was  discovered  during  final  acceptance  tests  at 
the  factory. 

The  TFR  was  written  to  document  the  failure  of  a  spare  during  Installation 
and  testing  operations  before  deployment. 

The  TFR  was  written  to  document  the  failure  of  a  spare  during  Installation 
and  testing  operations  at  the  shipyard  before  deployment. 

The  calendar  or  operating  life  of  the  component  was  exceeded,  or  the 
remaining  calendar  or  operau.,g  life  was  insufficient  for  deployment. 

The  word  "Life"  is  to  be  modified  by  a  subfield  as  follows:  Lit;/ 
calendar  or  Life/operatlng . 

The  failure  occurred  or  was  discovered  during  SSBM  or  Tender  overhaul 
operations.  This  category  also  applies  to  items  replaced  although  not 
specifically  considered  failed  components. 

The  TFR  was  written  to  document  the  failure  of  a  spare  during  overhaul 
operations . 

The  TFR  refers  to  components  failed  or  replaced  In  compliance  with 
specific  preventive  maintenance  programs.  The  program-identifying 
number  must  be  included  as  Indicated. 

The  TFR  refers  to  components  returned  on  compliance  with  specific  SP 
recall  directives.  This  type  of  TFR  will  seldom  be  a  valid  failure,  and 
In  most  cases  the  Item  Is  returned  for  special  Investigations  and  the 
word  "Recall"  appears  on  the  TFR. 

The  failure  occurred  or  was  discovered  during  the  Intlal  receiving 
Inspection  and  test  operations  on  new  components  received  by  shore 
activities  directly  from  the  vendor,  such  as  missile  components  at 
PMFA/tCPB . 

The  component  did  not  fail  but  for  one  reason  or  another  was  returned 

for  recertification.  The  word  "Recertification"  appears  on  this  type  of  TFR. 

The  failure  occurred  or  was  discovered  during  refitting  operations  per¬ 
formed  as  part  of  the  refit  program.  The  word  "Refit"  appears  on  this 
type  of  TFR.  This  category  Is  used  on  PMFA  or  PMFP  TFRs  on  missile 
components  in  the  same  manner  as  the  category  Overhaul"  fur  SSBN 
and  Tender  equipment . 

The  trouble  cr  failure  occurred  or  was  discovered  during  reussting  or 
repair  of  fleet  returned  failed  components. 

Tne  TFR  was  written  to  document  the  repair  of  a  failed  spare  by  a  repair 
activity. 

The  trouble  or  failure  occurred  or  was  discovered  duilng  normal  routine 
operations  and  tests  on  board  SSBNs  and  Tenders.  This  category  will 
also  apply  to  failures  of  test  and  checkout  equipment  during  normal 
operations  at  shore  stations. 


'  v. « 
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TABLE  5  (Cont) 


Type 

Description 

SPALT/0000 

The  TFR  was  written  to  document  difficulties  encountered  in  the  ac¬ 
complishment  of  an  authorized  alternation  (SPALT,  SHIPALT,  3PALTRA, 
ORDALT,  WEPALT),  or  to  report  inadequacies  in  materials  or  Instructions 
provided  for  the  accomplishment  of  alterations. 

Spare 

The  failed  component  was  a  spare  part,  e.g. ,  a  defective  spare  which 
was  removed  from  stock  for  Inspection  and  test.  This  category  is  most 
applicable  to  SSBN  and  Tender  spares.  It  is  not  a  useful  category  for 

PMFA  cr  PMFF  TFRs  since  many  components  delivered  to  these  activities 
are  not  always  assigned  to  a  system,  and  therefore,  all  such  units 
couio  be  erroneously  considered  spares. 

Special  test 

The  failure  occurred  during  compatibility  tasting  aboard  Tenders  or 

SSBNs  or  at  shore  activities.  This  category  includes  the  testing 
operations  performed  on  SSBNs  in  the  shipyards  which  cannot  be  classed 
as  routine  or  installation. 

Surveillance 

The  failure  occurred  while  performing  inspection  and  testing  on  com¬ 
ponents  in  storage  in  accordance  with  a  surveillance  program. 

General  Failure  Categories  (Subfields) 

Functional 

Any  unsatisfactory  performance  during  testing  or  operation  or  any  non¬ 
conformance  to  operating  requirements.  It  is  to  be  used  to  modify  the 
"Type  of  TFR"  categories  if  applicable. 

Handling 

Any  defect  that  can  be  attributed  to  handling  by  Navy  personnel. 

Unconfirmed 

Any  previously  reported  failure  not  verified  by  the  subsequent  activity 
retesting  the  component.  It  is  to  be  used  to  modify  the  repair-type 
category  only,  if  applicable,  as  follows:  repair/unconfirmed. 

Workmanship 

Any  defect  which  can  be  attributed  to  the  oversight  or  carelessness 
of  factory  personnel. 

Design 

Any  defect  or  unsatisfactory  performance  reported  on  the  TFR  as  a 
design  deficiency  (not  an  analysis  by  FMSAEG). 

Shipyard 

Any  defect  which  is  reported  on  the  TFR  as  caused  by  an  oversight  or 
by  carelessness  of  shipyard  personnel. 
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TABLE  6 


STANDARD  FAILURE  DESCRIPTOR  PHRASES 


Adjustment/improper 
Alignment/ improper 
Arcing/'exces  sive 
Back  re s i ;  ance/low 
Balance/improper 
Bandwidth/  narrow 
Bias/high 

Calibration/  improper 
Capacitance/low 
Circuit/open 
Clearance/ improper 
C  ont  am  inati  on/oil 
Crimping/inade'quate 
Current/excessive 
Cycling/ improper 
Fatigue 

Feedback/ improper 

Fit/loose 

Flooded 

Frequency/ drifting 
Friction 


Gain/low 

Humidity/ exce  s  sive 
Identification/ mis  sing 
Idle  current/iow 
Impedance/mismatch 
Index 

Inductance/low 
Insulation/  split 
Jitter 

Leakage/ nitrogen 

Lubrication/inadequate 

Null 

Operation/improper 
Output/ noisy 
Packaging/  improper 
Part/missing 
Phase/  shifted 
Polarity/ reversed 
Positioning/ improper 
Power 

Precharge/low 


Pressure/low 
Regulation/  poor 
rp  m/low 

Residual  magnetism/ 
lacking 

Resistance/low 

Response/slow 

Roll/ improper 

Sequence/ incorrect 

Sensitivity/low 

Soldering/poor 

Staging/ improper 

Surface/scratched 

Switching/ incorrect 

Temperature/low 

Tolerance /low 

Transducer  excitation/ 
low 

Unknown 
Voltage/low 
Wear/ excessive 
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detailed  narratives;  however,  all  indications  are  that  human-initiated  malfunc¬ 
tions  are  seldom  found  in  the  reports.  The  closed-loop  system,  with  contractor 
participation  through  CARs,  should  provide  a  vehicle  for  identifying  human- 
initiated  failures,  but  experience  proves  otherwise. 

Surface  Missile  System  and  Air  Launch  Missile  System  Performance  and 
Failure  Reporting 

3.24  Performance  and  failure  reporting  as  related  to  these  two  categories 
of  missile  systems  are  also  under  the  cognizance  of  FMSAEG  in  Corona, 
California.  The  type  of  information  available  is  essentially  the  same  in  both 
areas.  A  detailed  analysis  is  performed  in  conjunction  with  each  firing  and  a 
considerable  amount  of  information  is  compiled  primarily  from  firing  reports 

and  telemetry  data.  Because  of  the  general  similarities  between  the  two  system 
categories,  only  two  basic  programs  are  described  in  detail  in  this  report; 

(a)  Air-Launch  Guided  Missile  Systems  Performance  Data  Reporting  Program, and 

(b)  Surface  Missile  System  Status  Log. 

3 . 25  Air-Launch  Guided  Missile  Systems  Performance  Data  Reporting  Program . 
The  purpose  of  the  program  is  to  collect  air-launch  weapon  system  data  from 
representative  fleet  squadrons  under  day-to-day  operating  conditk  ,»s.  Weapon 
system  performance  data  reports  are  collected  routinely  on  all  air-launch 
missile  firings  and  attempted  firings  and  on  all  operations  involving  counter¬ 
measures  which  attempt  to  degrade  system  performance.  Special  performance 
data  are  also  collected  as  required  on  a  sampling  basis  to  meet  requirements 

not  satisfied  by  routine  reports. 

3.26  The  data  collection  program  operates  as  follows .  Routine  report 
forms  (NAVWEPS  Forms  8811/4,  8811/5,  8811/6,  and  8o  12/1 ,  as  applicable) 
are  completed  each  time  an  air-launch  missile  firing  attempt  is  made,  or  each 
time  the  weapon  system  is  employed  in  a  countermeasure  environmer.;,. . 

Examples  of  these  forms  are  shown  in  Figures  8  through  10,  To  provide 
necessary  data  on  subsystems  or  operations  not  obtained  routinely,  FMSAEG 
administers  indiviaual  sampling  programs  in  specific  problem  areas.  In  most 
instances,  these  programs  require  highly  individualized  data  forms  in  lieu 

of  the  routine  data  forms  mentioned  previously.  Figure  11  presents  the 
code  sheets  used  in  conjunction  with  the  report  forms. 

3.27  With  the  aid  of  telemetry  data,  in-deplh  analysis  can  be  performed 
and  the  performance  of  the  systems  under  test  evaluated  with  reasonable 
accuracy.  The  analysis  of  the  results  of  these  findings  have  revealed  that 
up  to  10  percent  of  all  the  failures  can  be  attributed  to  the  human  element. 
Indications  are  that  this  figure  Is  considerably  lower  than  the  actual 
percentage.  Unfortunately,  outside  of  pointing  to  the  significance  of  the 
problem,  this  type  of  information  is  of  little  use  to  the  human  reliability 
analyst.  There  has  been  virtually  no  attempt  made  to  analyze  the  human- 
initlated  failures  to  try  to  relate  them  to  personnel  characteristics.  The 
night  Crew  History  shown  in  Figure  12  could  be  used  for  more  meaning¬ 
ful  analysis  by  comparing  crew  characteristics  with  failures. 
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(  Pages  1  and  2) 
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*  -  At  RCA  A*  T  MOT  L  AUNCMIO 

14.  REASON  FOR  DEGREE  COMPLETED 

A  -  tAT  CON*lCTtOM 
A  -  T«  OR  INlTRUNfNf  TROLlALI 
C  -  Mi  ILL  I  TAOUALC 
|  -  LAUMCHIR  TAOUALC 

*  -  CAMCfLLf 0  AT  CONMAMO 
G  -  Al  RAO A A  TAOUAL I 

N  -  TARGIT  TAOUALC 
4  -  CW  T  AAM1MITT  |  R  TWO  UAL  I 
A  -»R  Ml  TROUALI 
H -  COM*VT|R  TAOUAL I 

*  -  •  AOlO  AHB  C0MRWN»CATK)N  TAOUAL  I 

*  -  4-C  TAOUAL!  lOTMf  A  TMAM  AMC1I 
T  -  Alt  CONTROL  TROUAL  C 

I  .  OTHIR  (|1*L AINI 

B  AIRCRAFT  LANOING  CONDITION 
A  -  u* 

DOM  AC  CALI  II  or 
A  .  aiAAAamC 
c  -  0  ROM  AMCC 
I  -  IH C TRONIC! 

*  -  *OW|R  ALAMT 
»  .  MULTWLl  (|R*t  AIN* 

■  -  OTNJA  (|  l*L  A'Ml 


INTERCf  PT  DATA 


S.  BOGEY  TYPE 

1 

aircraft  types 

OTHIR  THAN  A  C  TYPES  j 

A  .  A4 

)  -  DCLMaR  | 

A  -  * < A,  *4C 

J  -  AON-JTA  | 

C  -  AJA 

4  -  OAAA 

1  *  -  FA 

1  -  ROCO 

A  -  *  lit.  Alt*.  Aiat 

•  -  MVAA 

c  -  A  lit.  A  IDA 

T  -  A0M-J4 

M  -  in.  ICIM.  TAT.  DC« 

K  -  (AAA 

1  .  ROM.  >4  A 

L  -  t  A1AA 

10-  TOU-22A/M 

*  -  (IA.  f  JA 

X  -  OTHEN  (EXPLAIN) 

A  -  A  JA 

T  -  AS 

X  -  OTMfR  If  l*LAH4l 

S  •  SUP  A  ACC  (EXPLAIN) 

MALFUNCTION  ANALYSIS  -  POST  FLIGHT 


«A.  MISSILE  MALFUNCTON  AT  LAUNCHER  MALFUNCTION 

A  -  CNICKt  tAT  A -CNfCKI tAT 

A  -  IGMITIA  |a A |.  arm  A  -  MOTOR  *IAI  L(AO 

MOT  ARMlOft*<m  IMARORIRLT  COMNICTIO  It* *l» 

C  -  tAACTT  AIN  MOT  RIMOVCO  C  -  UMAILICAL  MOT  R|TRaCT|0 

(1*1111 

CNfCKI  UNtAT  IVlt  AOLLO0IMC  ■  -  LAUNCH |R  CMICKt  UNIAT 

coon  aho  ik*l  aim  r**r  o*  iiialkimi 

Tf  JT  fOLf*MfMT  UlfOl 

I  .  hi  aO  mtqRauliCS  AS  AMCS  MALFUNCTION 

lTRCAM|Olt*lll)  A.  CMICKt  tAT 

*  -  |*U-  AATTIRT  AIRID  A  -  VIATICAL  G«R0 

lt*ni»  6  -  Al  Radar  troualc 

G  .  M*U  AlRCD  It*  Hit  1  -  C*  ILLUMINATOR 

M  .  CAt  GRAIN  *IAtO<V*l  K  .  IA  HT  TROUALI 

K  .  MiTROCfM  ARC  ttURI  H  -  COMRuTf  R  TROD  AL  I 

TROUALI  (W-IO  T  -  MULTIALI  If  1*LAIM» 

T  .  MULTIALI  IlKALAtN)  I  -  OTHfR  If  t*LAINt 

■  -  01N«  RI|I*L  A«N> 


MISSILE  IDENTIFICATION  DATA 

TO  GIVE  RUN  NUMtEROR  CODE  NUMBER  (BELOW)  ON  WHICH  TONE  PAS 
CHIC*  ED  UN  SAT. 

1  .  ALL  RUN* 

•  -  OCCK  OML  T 
I  -  OTMfR  <1  l*LAMi) 

71  VAlRiCAO  TYPE  -  (SPARROW  III  AND  SIDEWINDER) 

A  -  •  ■  ItCltC  ftAOTTiMC  CmaRGI I 
A  -  LIVl  IIRALOtlVft 
C  -  IMIRT 
f  -  r» 

1  -  OTMCR  (|K*lAIM) 

Tl  73.  A  75  SPARROW  HI  SELECT  LIGHT  AND  RE  AO  Y  LIGHT  ANO 
S' Of  WINDER  TONE 

A  -  T*i 

A  -  MO  lAAILUAO 
C  -  tNTIRMJTTfMT 
C  -  TftONOfCK  NO  IN  All 
A  -  TUMfO  U*  LATf 
C  -  MPT  CHtCKfO 

■  .  OTNfR  If  l*L  A«Mi 


7*  FUZE  TYPE 

A  .  MK  JM 
A  -  Ml  JJ1 


IIMClPMUIC-MM/)  (t  t  -Ml 


FIGURE  11.  AIR-TO-AIR  MISSILE  WEAPON  SYSTEM  FLIGHT 
REPORT  CODE  SHEET 
(Page  1) 


11HP.EMSAgft.HI/S  0-44) 


COOI  SHEET  FOE  TYPE  II  (PI,  A4,  A6  TYPE  A/C) 

Atlt.TO.Alft  WEAPON  SYSTEM  PLIGHT  REPOftT 


COOCS  OH  THIS  SHEET  MAY  BE  ENTERED  OK  THE  AAMBEP  FORM  PRIOR  TO  OR  IMMEDIATELY  FOLLOWING  THE  FLIGHT.  OTHER 
COOES  WILL  APPEA*  OIRICFLY  OH  THE  AAMREP  FORM. 

NOTE; 

U$a  o  doth  ( -  J  In  mil  coamt  In  wM/ch  >K*  Itmm  la  n«f  oppf/coW*. 

SQUADRON  DATA 

4. TYPE  OF  EXERCISE 

12.  MISSION  PROPOSEO 

14.  REASON  FOR  DEGREE  OF  COMPLETION 

A  ■  AAftEX 

NON-WEAPON  SYSTEM  TYPE 

A  .  SAT  COMPLETION 

B  .MISSEX 

S  •  ALL  NON-WEAPON  TYPE 

B  •  TM  OR  INSTRUMENT  TROUBLE 

C  -LANT  MISSEX 

C  -  MISSILE  TROUBLE 

E  -WESTPAC  MISSEX 

WEAPON  SYSTEM  TYPE 

E  -  LAUNCHER  TROUBLE 

F  -MIOPAC  MISSEX 

A  -  SPARROW  III  LAUNCH 

F  .  CANCELLED  BY  COMMAND 

G-  EASTPAC MISSEX 

B  ■  SIDEWINDER  LAUNCH 

G  •  Al  RAOAR  TROUBLE 

H-  MED  MISSEX 

C  •  SP  III  ♦  S/W  LAUNCH 

H  -  TARGET  TROUBLE 

X  .  PREWEP  TRAEX 

E  .  AIR-TO-AIR  SUPERIORITY 

J  -  CW  TRANSMITTER  TROUBLE 

L  -  WEPTRAEX 

F  •  COMBAT  AIR  PATROL 

K  -  IR  SET  TROUBLE 

P  -COMPEX 

G  *  ESCORT  FIGHTER 

M  -  COMPUTER  TROUBLE 

R  •  NORMAL  training 

H  .  INTERCEPT  TRAINING 

P  -  RADIO  &  COMMUNICATION  TROUBLE 

T  -  0.  R.  1 

X  •  SPEOAL  WEAPONS 

R  .  A/C  TROUBLE  (OTHER  THAN  AMCS) 

X  •  OTHER  (EXPLAIN) 

L  •  RESEARCH  AND  DEVELOPMENT 

T  -  AIR  CONTROL  TROUBLE 

P ■ SCRAMBLE 

X  -  OTHER  'EXPLAIN) 

t.  FLIGHT  GEAR 

R  *  CM  FLIGHT  TEST 

A  .  NORMAL 

X  .  OTHER  (EXPLAIN) 

17.  AIRCRAFT  LANDING  CONDITION 

B  •  EXPOSURE  SUIT 

A  •  UP 

C  PRESSURE  SUIT 

13.  MISSION  COMPLETION 

A  -  COMPLETED 

DOWN  BECAUSE: 

II.  type  of  controller 

B-  PARTIALLY  COMPLETED 

B  -  AIRFRAME 

a.  MISSILE  RANGE.  RADAR 

C.  NOTHING  COMPLETED 

C - ORONANCE 

B  ■  SELF  CONTROLLED 

E  -  AIRCRAFT  NOT  LAUNCHED 

E  -  ELECTRONICS 

C-CIC 

F  -  POWER  PLANT 

E  ■  GCI 

T  -  MULTIPLE  (EXPLAIN) 

T- MULTIPLE  (EXPLAIN) 

X  -  OTHER  (EXPLAIN) 

X  -  OTHER  (EXPLAIN) 

DATA  LINK 
F  •  ATDS 
G -  SAGE 
M  -  MTDS 
J  •  NTDS 

VOICE 
K  .  MTDS 
L  -  NTDS 
M . SAGE 
P • ATDS 
R  •  WF-? 


MALFUNCTION  ANALYSIS  .  POSTFLIGHT 


54.  MISSILE 

A  •  CHECKS  SAT 
H  .  GAS  GRAIN  FIRED 
K  .  NITROGEN  PRESSURE  TROUBLE 
(S/w-IC) 

T  •  MULTIPLE  (EXPLAIN! 

X  •  OTHER  (EXPLAIN) 


35.  LAUNCHER 

A  .  CHECKS  SAT 
E  -  CHECKS  UNSAT  (EXPLAIN) 


56.  AMCS 

A • CHECKS  SAT 
G  •  Al  RADAR  TROUBLE 
K  IH  SET  TROUBLE 
M  •  COMPUTER  TROUBLE 
T  -  MULTIPLE  (EXPLAIN) 
X  •  OTHER  (EXPLAIN) 


FIGURE  11.  AIR-TO-AIR  MISSILE  WEAPON  SYSTEM  FLIGHT 
REPORT  CODE  SHEET 
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PROJECT  SPARROW  SHOOT  -  FLIGHT  CREW  HISTORY 


1.  DATE 


2.  SQUADRON  3a.  PILOT  (Full  Name) 


3b.  RIO  (Full  Name) 


HHi 

HHI 

■■IB 

■  ■ 

■Bi  | 

■■■i 

HBBI 

II 

■BBS 

II 

bbbm 

IgaBftB 

■bbb 

|| 

bbbb 

IffTiiTI 

^^■Bl 

■  I 

■ 

hh 

hh 

1 

Ml 

■mbi 

1  ■ 

ggyigti 

■Bi 

bbbb 

■MB I 

4.  mSAEC  Assigned  Number 


[FLIGHT  TQg 


5.  Total 

6.  F4  Type 

7.  Jet 


8.  R.A.G.  (Replacement  Air  Group) 

9.  East  Coast  (VF-101) 

10.  Weet  Coast  (VF-121) 

11.  No  Formal  R.A.C.  Training 

12.  RAG  Grad  Within  6  months 

13.  RAG  Grad  Within  1  Year 

14.  RAG  Grad  More  Than  1  Year 

13.  Years  Designated  Naval  Aviator /RIO 

16.  Age 

NUMBER  OF  PREVIOUS. 


17.  Day  Fighter 

18.  All  Weather  Interceptor 

19.  V.A. 

20.  Other 

TOTAL  CARRIER  ARRESTED  1ANDIHCS 


21.  In  Type  (K4) 

22.  Total  Jet 

23.  Total  -  All  Models 

CREW  EXPERIENCE  (Approximately) 


24.  Total  Hours  Crew  Has  Fleam  Together  As  A  Crew 

25.  Total  Hours  Crew  Has  Flown  Together  In  Last  6  Months 

26.  Average  Number  Intercepts  Flown  Per  Month 

27.  Total  Nusiber  Intercepts  Flown 
NUMBER  Of  MISSILES  FIRED 

28.  AIM-7  Series 

29.  AIM- 9  Series 

30.  Data  of  Last  AIM-7  Firing 

31.  Pilot/RIO  As  A  Team  (AIM-7) 

32.  Pllot/RIO  As  A  Team  (AIM-9) 


-I 


233-9l-(2-66) 

FIGURE  12.  FLIGHT  CREW  HISTORY 
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3.28  Figure  13  shows  a  Firing  Report  which  is  used  in  conjunction  with 
Surface  Missile  Systems.  The  analyses  performed  on  these  systems  do  not 
appear  to  be  as  detailed  as  those  performed  for  the  Air-Launch  Systems. 

3.29  Surface  Missile  System  Equipment  Status  Log.  This  system  was 
established  for  reporting  the  operational  status  of  the  nonexpendable  equip¬ 
ment  of  Surface  Missile  Systems.  The  reporting  system  is  designed  to 
enable  the  data  to  be  correlated  with  the  Maintenance  Data  Collection  Sub¬ 
system  (MDCS)  which  is  part  of  the  3-M  system.  Report  8821/5  (Figure  14) 
is  submitted  weekly  giving  the  status  of  the  equipment  for  each  day  of  the 
week  and  also  each  time  there  is  a  change  in  the  status.  Table  7  provides 
the  definitions  of  the  various  status  categories. 

3.30  From  these  reports  from  all  ships,  system  availability  is  determined 
based  on  the  percentage  of  time  the  system  was  up  and  the  percentage  of 
down-time.  There  is  no  indication  of  how  accurately  the  status  of  the 
equipment  is  assessed  and  reported  at  any  given  time  aboard  ship.  Systems 
availability  could  be  used  by  relating  it  to  crew  composition.  This  type 

of  analysis  is  similar  to  those  outlined  in  Section  IV. 

3-M  System* 

3.31  It  is  difficult  to  discuss  the  operations  of  the  3-M  system  because 
it  is  still  in  a  stage  of  development;  consequently,  changes,  modifications, 
and  additions  are  incorporated  quite  frequently.  The  following  discussion  takes 
Into  consideration  the  most  recent  changes. 

3.32  The  3-M  system  consists  of  two  subsystems:  The  Planned  Mainten¬ 
ance  System  (PMS)  and  MDCS.  PMS  is  designed  to  provide  procedures, 
schedule  phasing,  manpower  plans,  and  material  requirements  for  preventive 
maintenance.  MDCS  is  established  to  report  extensive  data  on  corrective 
maintenance  transactions. 

3.33  MDCS  is  the  only  portion  of  the  3-M  system  that  can  provide 
data  in  support  of  human  reliability  analysis .  MDCS  is  designed  to  pro¬ 
vide  a  means  of  recording  maintenance  actions  in  substantial  detail  so 
that  a  great  variety  of  information  may  be  retrieved  concerning  maintenance 
actions  and  the  performance  of  equipment  involved.  In  addition  to  record¬ 
ing  maintenance  actions  performed,  the  system  provides  data  concerning 
the  initial  discovery  of  the  malfunction,  how  equipment  malfunctioned,  how 
many  man-hours  were  expended,  which  equipment  was  involved,  what  repair 
parts  and  materials  were  used,  what  delays  were  incurred,  the  reasons  for 
delay,  and  the  technical  specialty  or  rating  of  the  person  who  performed  the 
maintenance. 

-Jc 

Much  of  the  information  on  the  3-M  system  was  taken  from  the  3-M 
Manual,  OPNAV43P2.  15/ 
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FIGURE  13.  FIRING  REPORT  FIGURE  14.  NON-EXPENDABLE  SMS  EQUIPMENT 

STATUS  REPORT 


TABLE  7 


STATUS  CATEGORY  DEFINITIONS  AND  CODES 


Status  Category.,  Definition 


Operating  Full  or  Reduced  Capability.  Equipment  is  energized  and  is  being 
operated  at  full  or  reduced  capability. 

a.  Operable  at  Full  Capability  -  Capable  of  operation  with  no 
known  impairments  or  degradations . 

b.  Operable  at  Reduced  Capability  -  Having  known  impairments 
or  degradations,  but  still  capable  of  operation  and  of  accom¬ 
plishment  of  a  significant  part  of  the  tactical  mission. 


Ready  or  Standby .  Equipment  is  partially  energized  and  is  believed  to  he 
operable  at  full  or  reduced  capability.  For  seme  equipment,  full  power  may 
be  applied  but  the  operate  or  transmit  switch  is  not  activated. 

Secured .  Equipment  is  not  energized  but  is  believed  to  be  operable  at  full  or 
reduced  capability. 


Inoperable  -  Active  Maintenance.  The  equipment  is  not  capable  of  operating 
at  full  or  reduced  capability  on  demand.  The  equipment  is  undergoing  correc¬ 
tive  or  preventive  maintenance. 


Inoperable  -  Waiting  Spares.  The  equipment  is  not  capable  of  operating  at 
full  or  reduced  capability  on  demand.  The  equipment  cannot  be  restored  to 
an  operable  status  ur.t.l  receipt  of  needed  parts. 


Inoperable  -  Modifications.  The  equipment  i-.  not  capable  of  operating  at 
i  full  or  reduced  capability.  The  equipment  cannot  be  restored  to  an  operable 
status  until  completion  of  the  modifications. 

Inoperable  -  CKitside  Help.  The  equipment  is  not  capable  of  operating  at  full 
or  reduced  capability  on  demand  and  required  the  assistance  of  outside  per¬ 
sonnel  and/or  equipment  to  restore  it  to  an  operable  status. 

Inoperable  -  Administrative.  The  equipment  is  not  capable  of  operating  at  full 
or  reduced  capability  on  demand.  Ship’s  operations,  activities,  or  procedures 
prohibit  maintenance  action.  Equipment  is  inoperable  hut  no  maintenance  is 
being  performed. 


Inoperable  -  Support  Equipment.  Tne  equipment  is  not  capabh  of  operating  at 
full  or  reduced  capability  on  demand  because  of  the  failure  of  some  other  p'cc* 
of  equipment,  such  as  power  equipment,  test  equipment,  switchboards,  dehy¬ 
dration  equipment,  or  pressurization  equipment. 
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3.34  The  MDGS  provides  a  document  on  which  maintenance  personnel 
record,  at  the  source,  designated  information  concerning  planned  or  corrective 
maintenance  actions.  The  information  is  recorded  ^  »  coded  '"onfiguration 
which  permits  machine  processing.  Each  maintenance  action  is  reported 

in  this  manner.  Copies  of  the  documents  prepared  by  supply,  personnel 

for  Issuing  parts  resulting  from  these  maintenance  actions  provide  material 

and  cost  information  to  the  Maintenance  Data  Collection  Center  for  each  activity. 

3.35  Routine  preservation  (chipping,  painting,  and  cleaning)  and  daily  or 
weekly  Planned  Maintenance  actions  are  reported. 

3.36  Documentation  in  the  MDCS  is  accomplished  by  the  completion 

of  one  or  more  forms,  as  applicable,  aboard  ship.  The  following  describes 
the  three  most  significant  forms. 

a .  OPNAV  Form  4700-2B,  Shipboard  Maintenance  Action, 

is  used  to  record  the  completion  of  planned  maintenance 
actions,  corrective  maintenance  actions,  and  authorized 
alterations  that  have  been  performed  at  the  shipboard 
level  by  shipboard  personnel.  All  planned  maintenance 
actions  except  daily  and  weekly  preventive  maintenance  are 
recorded  (see  Figure  15). 

b.  OPNAV  Form  4700-2D,  Deferred  Action,  is  used  to 
report  corrective  maintenance  actions  that  are  deferred 
due  to  ship's  operations,  lack  of  repair  parts,  or  the 
requirement  for  outside  assistance.  It  is  used  to 
record  the  reason  for  deferral  and  to  report  the  completion 
of  the  deferred  action.  From  this  form,  information  relating 

to  maintenance  action  deferrals  can  be  analyzed  (see  Figure  16). 

c.  OPNAV  Form  4700-2C.  Work  Request,  is  used  to 
document  the  need  and  re^-est  for  outside  repair  or 
"manufacture”  assistance.  It  is  used  for  workload 
planning  by  repair  activities  (i.e.  ,  tenders,  repair 
ships). 

3.37  Figure  1.7  presents  a  functional  flow  diagram  of  the  Shipboard 
Maintenance  Action  Report.  Tables  8  and  9  show  the  "How  Malfunctioned" 
and  the  "When  Discovered"  codes,  respectively;  and  Tabie  io  presents 
the  most  significant  "Action  Taken"  codes, 

3.38  All  data  provided  by  the  3-M  system  are  processed  through  ADP 
in  the  Mechanicsburg,  Pennsylvania,  facility  as  are  all  the  standard  and 
special  reports.  Since  all  the  information  is  on  tape,  it  is  more  appropriate 
to  present  here  all  the  data  elements  which  are  presently  punched  from  the 
source  document  rather  than  to  provide  a  listing  of  the  standard  reports. 

Table  11  presents  a  list  of  the  data  elements  punched  from  the  various  forms 
used  by  the  MDCS . 
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I.  MaiMeoonco  oction 
performed. 


HOW  MALFUNCTIONED"  CODES  “'WHEN  DISCOVERED"  CODES 


Description  ] 

When  Lighting  Off/Starting 

When  Securing 

During  Equipment  Operation 

During  Preventive  Maintenance 
(Requirement  of  MRC  Cards) 

Special  Inspections  (INSURV.  Material 
or  other  Requirements  Specified  by  Tech. 

Bureaus,  Flt/Tycoms) 

Underwater  Hull  Inspection 

During  Repair  or  Upkeep 

Upon  Receipt  from  Supply  Stores 

Unknown 

Not  Otherwise  Coded 

During  Development,  Test  It  Evaluation 
(For  Specified  Equipment  Only) 

Entered  Any  Time  the  Action  Taken  Is 

A  or  B  or  the  How  Mai  Code  Is  000 

© 

3 

ms 
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TABLE  11 
DATA  ELEMENTS 


Administrative  Organization 
Unit  Identification  Code 
Maintenance  Control  Number 
Date  (day-month-year) 

Type  Availability  Code 

Equipment  Identification  Code 

Work  Center 

Assisting  Work  Center 

Repair  Activity  Unit  Identi¬ 
fication  Code 

MAL/MRC 

When-Discovered  Code 
Action  Taken  Code 
Units 

Man-Hours  (tenths) 

Serial/  Noun 
Card  Code 

Alteration  Identification 
Equipment  Time 
Equipment  Down  Time 
Assigned  Work  Center 


Assigned  Assisting  Work  Center 

Estimated  Man-Hours 

Requesting  Work  Center 

Serial-Job  Description 

Desired  Completion  Date  (day- 
month- year) 

Service  Code 

Start  Date  (day-month-year) 
Source  Code 
Unit  of  Issue 

Unit  Identification  Code 

Maintenance  Control  Number 

Cog  Symbol 

Federal  Stock  Number 

Additional  (TSMC) 

CID/APL/AEL/AN 

Reference  Symbol 

Quantity- 

Unit  Price 

Card  Code  (FQl 


■* 
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3.39  The  3-M  system  is  continually  expanding  and  will  eventually 
cover  all  types  o'  systems  and  equipment.  Presently,  its  coverage  is 
most  extensive  in  the  electronics  area.  The  3-M  system  is  designed 
primarily  as  a  maintenance  accounting  system  providing  information  on 
material  and  manpower  usage  in  terms  of  quantities,  costs,  and  man-hours 
in  the  maintenance  area.  Although  the  system  does  contain  some  data  on 
failures  in  terms  of  types,  causes .  and  down  time,  the  coverage  in  this 
area  is  not  sufficient  for  in-depth  failure  analysis. 

3.40  This  coverage  is  particularly  insufficient  regarding  human- 
initiated  malfunctions.  In  its  present  operating  mode,  the  3-M  system 
has  no  provisions  for  identifying  human-initiated  failures.  The  only 
treatment  of  human  performance  deals  with  time  to  repair  which  can  be 
compared  with  a  previously  established  standard. 

3.41  It  is  significant  to  note  that  some  major  revisions  to  3-M  are 
presently  being  pilot-tested.  These  revisions  contain  certain  features 
which  will  have  an  important  bearing  on  the  failure  reporting  aspect  of 
the  system.  A  change  in  the  coding  system  for  identifying  failures  is 
being  tested  which,  if  incorporated,  will  result  in  a  significant  improve¬ 
ment.  First,  there  are  two  separate  coding  structures  established  to  dis¬ 
tinguish  between  failed-part  condition  (type  of  failure)  and  cause  of  failure. 
Further,  the  cause  of  failure  is  broken  down  into  three  categories:  envir¬ 
onment,  quality  of  part,  and  personnel.  Thus,  a  vehicle  is  being  provided 
for  identifying  human-initiated  malfunctions .  Tables  12  and  13  present 

the  primary  failed-part  condition  and  cause  code  structures. 

3.42  Unfortunately,  the  subcategories  under  personnel  are  not  par¬ 
ticularly  meaningful.  It  is  highly  questionable  that,  even  if  all  the  infor¬ 
mation  were  available  on  a  particular  failure,  "lack  of  skills"  could  be 
distinguished  from  "lack  of  training.  "  Also,  "maintenance"  and  "operating 
accident"  are  not  very  descriptive  phrases  for  identifying  a  human  error. 
However,  the  major  step  has  beer,  taken  in  separating  human-initiated 
failures,  and  some  of  the  more  specific  definitions  of  the  type  of  person¬ 
nel  error  will  evolve  through  actual  usage  and  further  analyses. 

Fleet  ASW  Data  Analysis  Program  (FADAP) 

3.43  FADAP  was  established  to  provide  an  evaluation  of  the  capabil¬ 
ities  of  present  or  future  ASW  forces  to  accomplish  ASW  missions.  A 
history  of  tactical  evolutions  and  current  ASW  exercises  provides  a  data 
base  for  this  evaluation.  This  data  base  can  be  used  for  exercise  recon¬ 
struction,  for  use  jn  programming  ASW  force  levels,  and  as  incuts  to 
associated  research,  development,  and  procurement  programs.  Long-range 
objectives  of  FADAP  are  to  satisfy  Navy  needs  for  ASW  operational  data, 

to  use  FADAP  data  for  training  and  readiness,  and  to  satisfy  ASW  analysis 
and  evaluation  capability  needs. 
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TABLE  12 


PRIMARY  FAILED  PART  CONDITION  CODE 


Code 


Description 


780 

Bent 

070 

Broken 

900 

Burned 

080 

Burned  out 

171 

Burred 

130 

Change  of  value 

910 

Chipped 

180 

Clogged 

027 

Collapsed 

160 

Contacts  conn,  defective 

170 

Corroded 

190 

Cracked 

479 

Crushed 

lib 

Cut 

200 

Dented 

117 

Deteriorated 

230 

Dirty 

145 

Dished/bulged 

231 

Elongated 

036 

Encapsulation,  faulty 

061 

Fused 

001 

Gassy 

Code 

Description 

428 

Incorrect  reading 

082 

Intermittent  open 

083 

Intermittent  short 

381 

Leaking 

004 

Low  GM  or  emission 

344 

Melted 

450 

Open 

003 

Open  filament 

429 

Peeled 

520 

Pitted 

964 

Pom  spectrum 

540 

Punctured 

935 

Scored 

Oil 

Screen  defects 

196 

Shorted  or  grounded 

018 

Test  OK,  did  not  work 

947 

Torn 

666 

Twisted 

628 

Wiped 

020 

Worn  excessively 

099 

Other  (submit  special  reporting 
form) 

TABLE  13 

PRIMARY  FAILED  PART  CAUSE  CODE 


Description 


Collision 

Corrosion 

Extreme  temperature 

Fire  damage 

Foreign  object  damage 

Malfunction  of  associated  equipment 

Shock 

Weather  damage 
Vibration,  excessive 
Wet 

Other  (submit  special  reporting  form) 


Quality  of  Part  (Procured  Malarial) 
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3.44  Exercise  data  collected  by  FADAP  are  organized  into  five  major 
files; 

a.  The  Exercise  Control  File  -ecords  background  param- 
?+ers  which  are  relatively  unchanging  throughout 
the  exercise. 

b.  The  Vehicle  Fitment  File  records  all  equipment  carried 
by  individual  aircraft  types  and  ships  participating  in 
the  exercise. 

c.  The  Exercise  Incident  File  records  critical  scenes  of 
actions  in  terms  of  "incident -defining  events . "  Three 
types  of  events  are: 

1 .  Valid  initial  detection 

2.  A  "no-detectlon"  event 

3.  A  false  contact. 

(This  is  the  key  file  in  the  data  library.) 

d.  The  LOFAR  File  contains  incident-defining  events  generated 
by  LOFAR  sonobuoys  and  SOSUS.  These  events  are  in  a 
separate  file  because  of  their  security  features. 

e.  The  Environment  File  contains  meteorological  and  oceano¬ 
graphic  conditions  associated  with  the  incidents. 

3.45  Data  are  initially  stored  in  the  ASW  and  submarine  force  fleet 
libraries  for  staff  analysis.  After  format  conversion,  they  are  forwarded 
to  the  Naval  ASW  Data  Center  which  was  established  in  Washington  in 
1965. 

3.46  One  of  the  difficulties  in  the  application  of  FADAP  is  the  enormous 
manpower  requirements  during  exercises  to  record,  reconstruct,  and  analyze 
incidents.  These  severe  demands  may  conflict  with  the  actual  operation  of 
the  exercise. 

3.47  The  data  elements  in  various  files  do  not  contain  data  easily  re¬ 
lated  to  human  errors.  The  alternative  reasons  for  "no-detection"  events 
are: 

a.  Insufficient  signal  for  recognition 

b.  Signal  masked  by  background  noise 

c.  Signal  masked  by  own  ship's  noise 

d.  Target  at  extreme  range 

e.  Own  ship  otherwise  tactically  engaged 
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f.  Sensor  equipment  impaired  or  inoperative 

g.  Environmental  interference 

h.  Other  (specify) 

i.  Unknown. 

3.48  None  of  the  alternatives  (except  possibly  h)  permits  the  admission  of 
human  error  as  the  cause  of  the  "no-detection"  event.  Recently,  however, 
the  ASWFORLANT  Scientific  Advisory  team  has  been  directed  to  become  the 
lead  research  team  in  the  development  of  the  FADAP  Human  Factors  File. 

PROBLEMS  IN  HUMAN-INITIATED  FAILURE  REPORTING 

3.49  One  underlying  problem  in  any  failure  reporting  system  is  that 
even  if  provisions  are  made  for  identifying  human  errors,  the  reporter 

is  reluctant  to  incriminate  himself  or  his  fellow  workers.  Unfortunately, 
this  reluctance  is  not  unfounded,  as  there  is  a  great  tendency  to  use 
these  reports  for  performance  evaluation,  especially  in  support  of  disci¬ 
plinary  actions  and  in  promotion  considerations .  Interviews  with  per¬ 
sonnel  associated  with  the  various  reporting  systems  revealed  that  "human 
error"  as  reported  by  these  systems  is  a  definite  factor  in  promotions . 

Specific  examples  were  cited  where  unfavorable  reports  on  individuals 
delayed  their  promotions  and  the  failure  report  was  used  as  evidence. 

Other  Incidents  were  mentioned  where  supervisors  refused  to  sign  any 
failure  or  malfunction  report  which  implicated  an  individual  or  the  crew 
as  a  whole. 

3.50  It  is  clearly  indicated  that  until  these  reports  are  used  only  for 
the  purposes  of  reliability  and  effectiveness  analyses  and  noi  for 
disciplinary  purposes,  there  is  little  hope  that  meaningful  data  can  be 
obtained  on  human  performance  or,  for  that  matter,  that  the  accuracy 
and  usefulness  of  any  data  for  analysis  of  equipment  performance  and 
reliability  can  be  depended  upon. 

3.51  Another  significant  problem  associated  with  the  failure  codes 
used  by  these  systems  is  the  number  of  possibilities  that  are  made 
available  for  the  reporter's  choice.  Most  systems  use  over  100  different 
codes  to  identify  a  failure  The  lists  of  these  codes  are  diluted  by  such 
non-descr!iptive  items  as  "broken,"  "inoperative,"  "failed-to-operate, " 
or  "deteriorated,"  which  are  certainly  not  very  informative.  There  is  a 
great  tendency  among  reporters  to  commit  to  memory  a  few  common  catch¬ 
all  codes  and  to  use  them  extensively  to  identify  failure 

3.52  During  the  course  of  this  study,  the  failure  history  over  a  period 
of  9  months  of  a  number  of  AN/SQS-2  3  Sonar  Systems  and  AN/SPS-40  Radar 
Systems  were  analyzed.  The  analysis  showed  that  of  94  different  codes 
available  to  identify  type  of  failure,  approximately  50  percent  were  never 


used  and,  of  those  used,  7  to  9  codes  identified  over  half  the  total  number 
of  failures.  Table  14  shows  the  results  of  the  analysis.  The  use  of  non— 
descriptive,  catch-all  phrases,  such  as  "broken"  or  "inoperative,"  is  a 
major  contributor  to  this  inaccuracy.  It  is  much  too  easy  for  the  technician 
to  apply  one  of  these  phrases  to  cover  a  multitude  of  situations  as  he  is 
filing  his  report.  Another  problem  in  reporting  systems  is  the  apparent 
interchangeable  usage  of  causes  for  failure  with  types  of  failure .  As  an 
example ,  "terminals  reversed"  indicates  why  a  system  or  subsystem  might 
have  failed,  whereas  "burned  out"  or  "blown"  indicates  how  the  equipment 
failed  or  malfunctioned . 


TABLE  14 

FAILURE  CODE  USAGE 


Systems 

Number  of 
Maintenance 
Actions 

Number  of 
Failure  Codes 
Available 

Number 
of  Codes 
Used 

Percentage 
of  Codes 
Used 

Codes  Used 
Accounting  for 
over  50  Percent  of 
Maintenance  Actions 

AN/SPS-40 

225 

94 

54 

57 

9  of  54 

Radar 

AN/SQS-23 

276 

94 

47 

50 

7  of  47 

Sonar 

3.53  Another  source  of  inaccuracy  associated  with  the  failure  codes 
used  by  various  reporting  systems  is  that  the  same  set  of  codes  is  used  for 
all  types  of  systems  and  equipments.  This  method  further  reduces  the 
ability  of  accurately  identifying  the  failure  by  the  selection  of  the  proper 
codes,  since  several  codes  have  no  relation  to  the  systems  in  question. 

A  significant  improvement  could  be  achieved  in  the  reporting  forms  if  the 
failure-identifying  codes  were  tailored  to  each  type  of  system  or  equip¬ 
ment  and  the  same  codes  could  be  retained  to  identify  common  failures. 

As  an  example,  a  missile  system  failure  report  would  need  certain  unlgup 
codes  that  would  have  no  application  to  a  communications  system  failure 
report. 

3.54  The  dual  role  of  operators  and  maintenance  personnel  as  reporting 
agents  poses  another  problem  in  obtaining  valid  data.  As  long  as  the 
possibility  exists  for  incriminating  oneself  or  fellow  workers  through 
failure  reporting,  the  probability  of  obtaining  accurate  information  will 
remain  very  low.  Industry  was  faced  with  a  similar  problem,  which  it 
solved  by  separating  quality  control  from  manufacturing.  The  possibility 
of  introducing  a  quality  control  agent  into  the  failure  reporting  effort 


1 


52 


should  be  Investigated.  The  operator  would  thus  be  relieved  of  the 
responsibility  for  determining  the  causes  and  types  of  failure  and  for 
preparing  a  failure  report.  This  function  would  be  performed  by  a 
quality  control  agent  who  would  ideally  belong  to  a  different  organi¬ 
zational  entity.  This  development  would  not  necessarily  result  in  an 
increased  number  of  personnel,  because  the  operators’  time  would  no 
longer  be  consumed  by  the  completion  of  failure  reports,  and  a  small 
number  of  them  might  therefore  be  replaced  by  quality  control  agents. 

These  agents,  of  course,  would  be  used  in  the  fleet  only  during 
peacetime  exercises,  but  shore-based  operations  would  not  necessarily 
be  limited  in  this  way.  This  concept  is  presently  being  practiced  in  many 
areas  where,  for  example,  contractor  personnel  perform  the  reporting 
function  or  prepare  an  independent  report  which  is  subsequently  used  for 
failure  analysis. 

CONCLUSIONS 

3.55  The  foregoing  discussions  led  to  the  previously  stated  finding 
that  existing  failure  reporting  systems  do  not  yiold  data  on  human-initiated 
malfunctions.  Most  of  these  systems  have  no  provisions  for  reporting 
Information  on  the  relation  of  human  performance  to  failures.  Many  of  the 
systems  do  contain  information  which  could  be  used  for  human  reliability 
analyses  which  employ  the  indirect  techniques  outlined  in  Section  IV  of 
this  report. 

3.56  The  problem  of  self-incrimination  associated  with  reporting  human- 
initiated  malfunctions  is  further  amplified  by  the  Navy’s  use  of  information 
obtained  from  these  systems  for  personnel  evaluation.  The  use  of  these 
reports  should  be  relegated  to  analysts  and  effective  systems  management. 

3.57  One  of  the  major  problems  in  existing  failure  reporting  systems  is 
the  forms  that  are  used  by  these  systems.  The  forms  often  contain 
superfluous  or  confusing  information  which  detracts  from  the  main  purposes 
of  the  reports.  The  coding  structure  identifying  types  and  causes  of 
failure  often  contains  several  nondescriptive  items  or  catch-all  phrases 
that  are  meaningless  from  the  standpoint  of  reliability  analysis.  The  same 
codes  are  used  for  all  categories  of  equipment  even  though  some  of  the 
codes  have  application  to  only  specific  equipments  or  systems. 

3.58  The  introduction  of  a  "quality  control"  agent  into  the  failuie  reporting 
cycle  might  make  the  reports  more  accurate  and  might  lead  to  the  inclusion 

of  information  on  human  performance. 

3.59  Specific  recommendations  for  improvement  of,  and  studies  dealing 
with,  human-initiated  malfunction  reporting  are  presented  in  Section  V. 
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IV.  INDIRECT  APPROACHES 


INTRODUCTION 

4.1  As  discussed  in  the  preceding  section,  reporting  of  human- 
initiated  malfunctions  poses  some  significant  problems,  some  of  which 
may  require  major  changes  in  operating  philosophy  and  to  existing 
reporting  systems.  However,  as  was  discussed  in  preceding  sections, 
without  meaningful  data  on  human  performance  human  reliability  will 
not  be  able  to  take  its  rightful  place  in  systems  effectiveness  considera¬ 
tions.  The  researcher  in  the  field  of  human  reliability  is  thus  faced  with 
the  dilemma  of  needing  data  from  the  "hardware  world"  but  discovering 
that  the  same  "world"  is  not  particularly  sympathetic  to  his  problem  of 
obtaining  the  required  information. 

4.2  ORI  has  developed  two  techniques  which,  in  the  final  analysis, 
complement  each  other  to  provide  meaningful  information  on  human  per¬ 
formance  which  can  then  serve  as  the  basis  for  human  reliability  analysis. 
These  two  techniques  are  referred  to  as  indirect  approaches,  becar.se 
neither  relies  on  the  direct  reporting  of  human-initiated  failures  and 
malfunctions.  Rather,  they  both  use  equipment  failure  data  and  infor¬ 
mation  on  the  composition  of  the  crew  which  operates  and  maintains 

the  equipment. 

4.3  One  of  these  techniques  is  termed  the  Elementary  Reliability  Unit 
Parameter  Technique  (ERUPT).  This  approach,  which  groups  the  components 
of  a  weapon  system  into  elementary  reliability  units  (ERUs),  provides  a 
means  of  inferring  two  human  performance  parameters  from  available  equip¬ 
ment  reliability  and  maintenance  data. 


Preceding  Page  Blank 
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4.4  The  second  approach  relates  certain  personnel  characteristics 
of  individuals  operating  and  maintaining  the  equipment  to  numbers  of 
failures  and  equipment  repair  times  by  the  application  of  multivariate 
correlation  analysis  techniques . 

4.5  During  the  course  of  this  study,  the  feasibility  and  applicability 
of  the  two  approaches  were  investigated,  and  mathematical  models  and 
equations  were  developed  for  their  application.  A  pilot  test,  on  a  limited 
data  base  was  also  conducted  during  the  development  of  the  two  techniques . 

4.6  The  remainder  of  this  section  describes  the  formulation  and  appli¬ 
cation  of  the  two  models.  The  appendices  to  this  report  contain  the  mathe¬ 
matical  development  of  the  models  and  the  detailed  results  of  the  pilot  tests. 

ERUPT 

4.7  ERUPT  quantifies  two  measures  of  human  performance  during  main¬ 
tenance  as  part  of  a  model  which  evaluates  system  readiness.  The  most 
significant  feature  of  the  technique  is  that  the  quantification  of  these 
human  performance  parameters  can  be  accomplished  by  using  equipment 
failure  and  maintenance  data  without  relying  on  human-initiated  failure 
reporting . 

Description  of  the  Model 

4.8  ERUPT,  for  human  reliability  analysis,  is  embodied  in  a  model  ex¬ 
pressing  weapon  system  readiness.  Two  of  the  parameters  that  comprise  the 
model  are  specifically  related  to  human  performance  during  maintainability. 

a.  a  -  the  probability  that  failure  is  detected 

and  repaired  during  maintenance 

b.  3  *“the  probability  that  maintenance  does  not 

induce  failure. 

4.9  The  model  consists  essentially  of  equations  and  computational 
routines  for  deriving  system  measures  of  effectiveness  from  failure  and 
maintenance  parameters.  Two  such  measures  of  effectiveness  hove  been 
defined . 

a.  Readiness  reliability.  Probability  that  the 
weapon  is  operable  at  the  time  of  Its  operating 
mission  or,  more  generally,  probability  that 
the  weapon  is  in  "go"  condition  when  it  is 
needed . 

b.  Mission-tactic  reliability.  Probability  that 
the  weapon  will  successfully  carry  out  a  given 
mission  with  a  prescribed  tactic, assuming 

the  weapon  is  ready  (operable)  at  the  beginning 
of  the  mission. 
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4.10  Of  these  two  measures,  only  readiness  reliability  was  considered 
in  this  research  effort,  since  only  the  two  human  performance  parameters. 

Of  and  8,  were  applicable  to  that  aspect  of  system  effectiveness.  The 
characteristics  of  the  weapon  system  used  tn  the  development  of  the 
technique  are  such  that  no  human  interaction  is  required  during  its  actual 
operational  phase.  However,  the  results  of  this  feasibility  study  clearly 
indicate  that  the  same  techniques  can  be  applied  with  some  modifications 
to  other  systems  in  all  phases  of  their  operational  usefulness. 

4.11  The  equations  for  readiness  reliability  are  given  in  Appendix  A. 

ERU 

4.12  One  of  the  most  important  concepts  associated  with  the  application 
of  ERUPT  is  the  grouping  of  system  components  into  ERUs.  The  selection  of 
ERUs  is  based  on  the  maintenance  level  established  for  the  system.  Main¬ 
tenance  level  in  this  context  is  defined  as  the  lowest  type  of  equipment 
indenture  at  which  maintenance  is  performed.  Level  of  indenture  is  a  term 
describing  the  breakdown  of  a  system  or  equipment  into  its  components. 
Thus,  if  the  first  level  of  Indenture  were  the  whole  equipment,  then  the 
second  level  would  be  the  components  that  make  up  the  equipment. 

4.13  It  is  significant  that  the  selection  of  ERUs  does  not  po^e  a 
problem  for  anyone  who  is  reasonably  familiar  with  the  system,  the  function 
of  the  various  components  or  subsystems,  and  the  maintenance  plan  for 
that  system.  The  latter  is  involved  with  the  throwaway  concept  which 
identifies  the  level  of  indenture  of  the  system  at  which  equipments  or 
components  wi'l  be  maintained  and  repaired  rather  than  replaced.  Failure 
of  an  ERU  Implies  no  useful  input  and,  therefore,  a  no-go  condition.  This 
ERU  failure  will  also  imply  that  the  system  is  in  a  no-go  condition. 

4.14  The  skill  level  of  the  maintenance  technician  is  also  related  to 
the  indenture  level  at  which  the  equipment  is  maintained  and  repaired. 

The  following  chart  illustrates  this  point.  The  shaded  areas  indicate  the 
indenture  levels  at  which  a  technician  of  a  given  ski1 1  level  can  be 
utilized. 


Level  of 
Indenture 


Skill  Level 


High 


Medium 


Low 


4.15  The  lower  the  indenture  level,  the  higher  the  skill  level  that 
will  be  required  on  the  part  of  the  technician.  He  will  be  required  to 
perform  more  detailed  and  complex  maintenance  and  repair  functions,  the 
more  detailed  the  component  he  repairs.  Conversely,  as  the  indenture 
level  is  raised,  the  maintenance  and  repair  activities  become  less  critical 
and,  consequently,  a  technician  with  a  lower  skill  level  can  meet  the  man¬ 
power  requirement,  although  a  higher  skilled  technician  could  still  be 
used. 

4.16  This  form  of  analysis  can  be  carried  one  step  further  to  include 
a  and  $.  Intuitively,  it  can  be  stated  that  the  probability  of  detecting 
and  repairing  a  failure  or  of  not  inducing  a  failure  will  increase  with  the 
skill  level  of  the  technician  and  will  increase  as  the  indenture  level  is 
increased.  As  the  indenture  level  increases,  the  maintenance  or  repair 
activity  is  less  complex  and  repairs  will  also  probably  occur  at  longer  time 
intervals.  Thus,  different  values  of  a  and  &  can  be  calculated  for  each  of 
the  nine  areas  on  the  chart. 

4. 17  This  type  of  analysis  can  have  a  major  impact  on  the  selection 
of  the  optimum  packaging  technique  to  be  employed  in  the  development  of 
a  new  weapon  system.  The  impact  of  different  packaging  concepts  on 

a  and  8  and  the  skill  levels  of  the  technician  can  be  readily  ascertained. 
This  approach  lends  itself  to  trade-off  analyses,  particularly  when  costs 
are  estimated  for  training  personnel  to  various  skill  levels  and  costs  are 
estimated  for  the  equipment  using  different  packaging  concepts.  Limiting 
factors  of  equipment  size  and  weight  dictated  by  usage  also  affect  the 
packaging  concept  and  thus  can  be  related  to  the  personnel  parameters . 

4.18  Readiness  reliability  for  the  weapon  system  is  the  product  of  the 
readiness  reliabilities  of  individual  ERUs.  The  formulation  of  ERU  readiness 
reliability  considers  the  impact  of  exercise  and  maintenance  on  an  ERU  until 
the  time  it  is  called  on  to  carry  out  a  prescribed  mission. 

4.19  The  model  for  the  calculation  of  ERU  readiness  reliability  functions 
by  first  using  ERU  failure  and  maintenance  data  to  estimate  values  of  the 
human  performance  parameters  (a  and  0).  Data  needed  are  as  follows: 

a.  Probability  that  the  ERU,  which  is  in  a  nonfailed 
condition  at  the  time  a  test  is  initiated,  survives 
the  test  (derived  from  laboratory  test  data) 

b.  Storage  time  between  planned  maintenance  of  each  ERU 

c.  The  mean  time  to  failure  established  for  each  ERU 
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d.  Number  of  maintenances  before  repair/replacement 
of  the  ERU  since  the  beginning  of  storage  or  since 
last  repair  of  the  ERU  if  corrective  malntenace  has 
been  done  previously. 

4.20  The  method  for  estimation  of  valuer  for  ot  and  ft  parameters  for  an 
ERU  is  based  on  actual  shipboard  failure  experience  and  maintenance  data 
for  the  ERU.  Essentially,  a  mathematical  function  involving  a,  8,  and  other 
hardware  parameters  represents  the  probability  that  the  actual  pattern  of 
failures  occurred,  given  various  values  of  the  parameters.  A  computer 
programmed  search  is  then  initiated  to  identify  the  set  of  values  for  a  and  ft 
which  would  maximize  that  probability.  That  set  of  values  is  the  maximum 
likelihood  estimate  of  a  and  ft.  A  complete  mathematical  formulation  of  the 
calculation  ot  the  human  performance  parameters  is  included  in  Appendix  A. 
The  general  thought  is  that,  if  "inherent"  or  "true"  hardware  component 
failure  rates  can  be  determined  under  controlled  conditions,  then  the  expected 
reliability  of  the  ERUs  can  be  calculated.  Then,  based  on  failure  rates  of 
these  ERUs  under  actual  conditions  obtained  from  maintenance  reports,  it 
will  be  possible  to  infer  the  o's  and  ft's  from  the  differences. 

4.21  With  the  values  of  a  and  8  just  calculated,  two  data  elements,  in 
addition  to  those  in  paragraph  4.19,  are  required  to  calculate  the  readiness 
reliability  for  each  ERU  at  any  time  following  manufacture.  The  additional 
data  needed  are: 

a.  Storage  time  of  the  ERU  between  last  planned 
maintenance  and  time  of  the  operating  mission 

b.  Number  of  tests  of  ERU  before  its  operating 
mission. 

4.22  Figure  18  shows  graphically  the  events  and  associated  probabilities 
that  comprise  the  life  cycle  of  an  ERU  until  the  lime  of  its  operating  mission. 
Arrows  show  the  various  times  that  the  ERU  can  go  from  a  nonfailed  to  a 
failed  condition  and  vice  versa . 

4.23  When  the  readiness  reliability  is  calculated  for  each  ERU,  the 
readiness  reliability  for  the  entire  weapon  system  is  found  simply  by  multir 
plying  the  ERU  readiness  reliabilities.  It  should  bo  emphasized  that  the 
equations  developed  for  readiness  reliability  are  applicable  to  any  weapon 
system  which  meets  the  underlying  assumptions  of  the  model. 

4.24  However,  it  also  is  true  that  the  readiness  reliability  measure  of 
effectiveness  is  not  the  only  measure  that  can  accept  ERUPT  human  relia¬ 
bility  parameters;  they  can  be  incorporated  equally  as  well  into  other 
models  of  system  reliability. 
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Test  Results 


4.25  Calculation  of  O'  and  8.  The  mathematical  formulation  that  provides 
statistical  estimates  of  a  and  8  (Appendix  A)  was  tested  by  using  input  data 
which,  although  not  from  an  operating  weapon  system,  are  believed  to  be 
realistic.  Appendix  D  shows  the  sets  of  data  used  with  the  formulations , 
together  with  values  of  aand  8  calculated  from  the  data.  Also  shown  is  a 
copy  of  the  computer  printout  for  the  first  set  of  data. 

4.26  In  interpreting  test  case  1  results,  a  total  of  five  different  samples 
(£M^)  of  shipboard  corrective  maintenances  showed  that  two  were  not  pre¬ 
ceded  by  preventive  maintenance  before  the  repair,  two  were  preceded  by 
one  preventive  maintenance  before  the  repair,  and  one  was  preceded  by 
two  preventive  maintenances.  The  best  statistical  estimates  of  aand  $. 
based  on  those  experiences  and  the  failure  distribution  shown,  are  a  -  0.98 
and  8~  0.33.  These  values  seem  reasonable  because  corrective  maintenances 
were  occurring  so  quickly  that  it  is  unlikely  that  malfunctions  were  being  . 
overlooked  (a)  to  any  great  extent.  However,  it  does  seem  possible  that 
malfunctions  were  being  induced  frequently  during  the  preventive  mainten¬ 
ances  {/?). 

4.27  Contrast  that  example  with  test  case  4  in  which  8  to  10  preventive 
maintenances  preceded  each  repair  or  corrective  maintenance  action.  In 
this  case,  it  seems  likely  that  the  malfunctions  were  not  being  detected 
promptly  during  preventive  maintenances.  This  is  reflected  in  the  low  0. 16 
value  of  a  estimated. 

4.28  The  input  specifications,  flow  chart,  and  computer  program  instruc¬ 
tions  for  the  estimation  of  a  and  8  are  given  in  Appendix  B. 

Sensitivity  Analysis  of  aand  8 

4.29  Under  a  separate  subcontract  with  the  Naval  Ordnance  Laboratory 
(NOL),  ORI  did  preliminary  work  on  the  sensitivity  on  system  readiness 
reliability  to  changes  in  aand  8  the  human  reliability  maintenance 
parameters.  The  assessment  was  in  support  of  the  development  program 
of  a  new  weapon  system. 

4.30  Significant  among  the  findings  was  the  fact  that  readiness  relia¬ 
bility  is  quite  sensitive  to  human  maintenance  parameters.  The  weapon 
system  consisted  of  275  ERUs  and,  to  bring  the  readiness  reliability  to 

an  acceptable  level,  It.  was  necessary  to  bring  the  probability  of  detecting 
a  malfunction  and  the  probability  of  not  inducing  a  malfunction  very  close 
to  unity. 
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4.31  Parameters  and  Assumptions  Used.  Table  15  shows  the  readiness 
parameters  used  in  this  assessment  and  the  assumptions  necessary  for  this 
preliminary  sensitivity  analysis. 

4.32  Table  16  presents  the  selected  values  of  the  model  input  parameters 
which  were  used  in  the  computations. 

4.33  In  Figure  19,  the  calculation  of  the  readiness  reliability  based  on 
various  combinations  of  these  values  is  plotted  against  a  ,  the  probability 
of  d^ecting  a  malfunction  which  fails  in  storage  or  test. 

4.34  As  shown  in  the  plot,  readiness  reliability  falls  off  rapidly  for  a 
near  0.9  as  0  moves  away  from  unity  and  ERU  storage  failure  rates  increase. 
However,  with  B  near  unity  and  low  storage  failuie  rates,  a  has  very  little 
impact  on  readiness . 

4.35  Comparison  of  curves  with  all  parameters  held  constant  except  8 
shows  the  large  impact  of  B  on  readiness  reliability.  A  decrease  in  ft  from 

1 ,0000  to  0.9990  causes  a  decrease  in  readiness  ranging  from  C.08  to  0.18 
depending  on  the  values  of  the  other  parameters.  This  fact  dictates  that 
there  must  be  unusual  emphasis  placed  on  the  attainment  of  values  of  B 
close  to  unity  for  complex  weapon  systems  with  a  large  number  of  ERUs. 

4.36  This  demonstration  clearly  shows  the  advantages  of  an  analytic 
model.  Such  sensitivity  analyses  can  show  which  of  the  parameters  is  most 
responsible  for  large  decreases  in  system  readiness  reliability  and  can 
provide  relative  quantitative  values  of  the  significance  of  each  of  the 
parameters . 

Conclusions 

4.37  The  ERUPT  approach  appears  to  be  feasible  for  human  reliability 
analysis  and  has  several  advantages  over  other  approaches  investigated. 

To  summarize  the  key  features  again: 

a.  It  does  not  require  direct  reporting  of  human 
errors  by  personnel  who  may  be  extremely 
biased 

b.  It  requires  only  hardware  failure  and  mainte¬ 
nance  data 

c.  The  data  required  are  simple  and  will  not 
necessitate  new  data  collection  studies 


d.  Based  on  the  specific  weapon  system  readi¬ 
ness  reliability  model  used  to  test  the 
feasibility  of  the  ERUPT  approach,  system 
readiness  reliability  is  extremely  sensitive 
to  values  of  /9  (the  probability  that  mainte¬ 
nance  does  not  induce  failure). 


TABLE  15 

READINESS  PARAMETERS 


Parameter 

Assumptions 

Storage  time  between  maintenances 

Same  for  all  ERUs 

Storage  failure  rate 

Fraction  of  functional  failure 
rate  for  electronic  ERUs; 
zero  for  mechanical  ERUs 

Probability  of  surviving  test 

Same  for  all  ERUs 

Probability  of  repairing  storage  or 
test  failures 

Same  for  all  electronic  ERUs; 
unity  for  mechanical  ERUs 

Probability  of  maintenance  not 
inducing  failure 

Same  for  all  electronic  ERUs; 
unity  for  mechanical  ERUs 

TABLE  16 


READINESS  RELIABILITY  INPUT  VALUES 


0 

TT 

t,  months 

1 

0.9990 

1.00 

X  /1000 

3 

■  :: 

1 .0000 

xj/ioo 

12 

a  =  probability  of  repairing  malfunction  in  ERU 
which  occurs  in  storage  or  test. 

$  =  probability  of  not  inducing  malfunction  in 

ERU  which  survives  storage  and  test. 

n  *  probability  of  ERU  surviving  test. 

V  »  storage  failure  rate  of  i  ERU. 

*  ^ 

=  functional  failure  rate  of  i  ERU. 

t  =  time  between  maintenances  in  months . 
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4.38  This  has  Indicated  that  meaningful  quantitative  informa¬ 

tion  on  human  performance  car.  be  derived  by  aji  indirect  method  of  hardware 
failure  and  maintenance  data.  The  ERUPT  approach  provides  a  basis  for 
meaningful  human  reliability  analysis  and  is  felt  to  have  wide  applicability 
to  present  and  planned  new  weapon  systems. 

MULTIPLE  CORRELATION  APPROACH 

Introduction 

4.39  Present  Navy  failure  reporting  systems  do  not  yield  data  on  human 
errors  or  human  performance.  This  general  conclusion  is  developed  in 
Section  III. 

4.40  The  multiple  correlation  approach  attempts  to  derive  some  useful 
information  on  human  reliability  from  present  systems.  This  approach  does 
not  attempt  to  provide  a  numerical  value  for  human  reliability  as  a  part  of 
system  effectiveness.  Instead,  the  technique  attempts  to  identify  personnel 
characteristics  which  show  the  greatest  effect  on  the  rate  of  equipment 
failure  and  repair  through  multiple  correlation  analysis . 

4.41  Multiple  correlation  analysis  is  a  well-known  statistical  technique 
used  to  measure  the  degree  or  importance  of  the  relationship  between  a 
dependent  variable  (or  criterion)  and  a  set  of  independent  variables  (or 
predictors).  The  relative  importance  of  each  of  the  predictors  can  be  assessed. 
Initially,  hypotheses  are  formulated  concerning  this  relationship  between 
dependent  and  independent  variables.  Multiple  correlation  analyses  then 
assess  the  interrelationships  between  these  variables. 

4.42  If  multivariate  correlation  analysis  can  identify  those  personnel 
characteristics  which  significantly  influence  equipment  failure  and/or 
mean  time  to  repair.  It  will  make  a  great  contribution  to  the  establishment 
of  requirements  for  recruitment,  training,  and  distribution  of  personnel. 

The  following  paragraphs  describe  the  approach,  show  the  results  of  a  pilot 
application,  and  make  recommendations  for  verification  of  these  results  and 
the  extension  of  the  technique  to  new  equipments. 

Description  of  the  Approach 

4.43  The  two  reporting  systems  presently  operating  in  the  Navy  and 
chosen  for  use  in  the  multivariate  analysis  approach  were  the  3-M  system 
and  the  Active  Duty  Enlisted  Master  File  maintained  in  Pers  19. 

4.44  The  3-M  system  can  provide  preventive  and  corrective  maintenance 
actions  data  for  specific  equipments  over  time  as  part  of  MDCS.  A  more 
complete  description  of  the  3-M  system  Is  presented  in  Section  III  of  this 
report. 
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4.45  Two  measures  of  equipment  failure  and  repair  were  identified  from 
3-M  reports:  first,  the  number  of  malfunctions  or  corrective  actions  taken 
during  specified  periods  of  time  and,  second,  the  mean  line  taken  to  repair 
these  malfunctions. 

4.46  Discussions  with  experienced  naval  personnel  led  to  the  formula¬ 
tion  of  hypotheses  concerning  personnel  characteristics  which  may  be  highly 
correlated  with  these  two  measures.  The  hypotheses  required  personnel 

dat  which  were  available  from  the  Active  Duty  Enlisted  Master  File  under 
Pers  19. 

4.47  The  Active  Duty  Enlisted  Master  File  contains  hundreds  of  personnel 
characteristics  related  to  the  identification,  qualifications,  education,  and 
assignment  of  all  active  duty  enlisted  personnel.  After  consideration  of  the 
hypotheses,  seven  personnel  characteristics  were  selected  for  use  from  this 
file. 

a .  Age 

b.  Pay  grade 

c.  Number  of  months  since  Active  Duty  Base  Dote 
(ADBD) 

d.  Number  of  months  until  Expiration  of  Active 
Obligated  Service  (EAOS) 

e.  Years  of  formal  education 

f.  Possession  of  Navy  Enlisted  Classifications  (NEC) 
pertinent  to  equipment  unde1'  study 

g.  Training  time  in  specialized  "C"  schools,  i.e.  , 

"C"  schools  directly  related  to  equipment  under 
study. 

4.48  The  AN/SPS-40  radar  and  the  AN/SQS  23B  sonar  were  selected  for 
the  feasibility  study.  The  selection  of  these  equipments  was  based  on  the 
availability  of  data  which  were  believed  to  be  reasonably  accurate  during 
the  time  period  chosen  for  the  test. 

4.49  The  AN/SPS-40  radar  is  operated  by  Radarmen  (RD)  and  maintained 
by  Electronics  Technicians  (ET) .  The  AN/SQS-23B  sonar  is  operated  and 
maintained  by  Sonar  Technicians  (ST). 

4.50  The  assumption  was  made  that  both  operators  and  maintenance 
personnel  affect  the  equipment  rate  of  failure  (number  of  malfunctions) 
through  their  errors.  Therefore,  personnel  data  fror.  both  the  RD  and  ET 
ratings  were  correlated  with  3-M  equipment  failures  for  the  AN/SPS-40 
radar.  Similarly,  personnel  data  for  the  ST  rating  were  used  to  test  for 
significant  statistical  relationships  with  the  number  of  malfunctions  for 
the  AN/SQS-23B  sonar. 

4.51  It  can  also  be  hypothesized  that  personnel  characteristics  of  j 

both  operators  and  maintenance  personnel  (RD  and  ET)  are  correlated  X 

with  mean  time  to  repair  for  the  AN/SPS-40  radar.  Similarly,  personnel 

1 
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data  from  the  ST  rating  can  be  correlated  with  mean  time  to  repair  for  the 
AN/SQS-23B  sonar. 

4.52  Data  were  acquired  for  both  equipment  and  personnel  parameters 
for  the  period  of  October  1965  through  September  1966.  They  were  subse¬ 
quently  grouped  into  3-month  intervals  because  itJwas  felt  that  this  period 
of  time  would  allow  significant  variance  in  the  values  of  the  parameters, 
and  would  also  permit  a  sufficient  number  of  sample  data  points  to  establish 
significant  relationships. 

4.53  Equipment  failure  and  maintenance  3-M  data  for  the  AN/SPS-40 
radar  were  available  for  the  three  quarters  ,  October-December  1965, 
January-March  1966,  and  April-June  1966.  Data  for  the  AN/SQS-23B  sonar 
were  available  for  the  same  three  quarters  plus  the  July-September  1966 
quarter. 

4.54  Equipment  maintenance  and  personnel  data  were  used  from  a  selec¬ 
ted  sample  of  ships  which  had  sonar  or  radar  equipment  on  board  during  the 
time  periods  selected  and  for  which  both  personnel  and  equipment  data  were 
available.  Table  17  Identifies  the  ships  used  in  the  sample  by  type  and 
hull  number. 

4.55  After  some  manual  data  tabulation,  equipment  failure,  repair,  and 
personnel  data  for  each  ship  in  each  3-month  period  were  correlated.  Thus, 
the  number  of  malfunctions  in  the  AN/SPS-40  on  the  DL  846  that  occurred 

in  the  January-March  1966  quarter  was  compared  with  the  personnel  charac¬ 
teristics  of  all  ETs  aboard  for  that  quarter.  The  number  of  malfunctions 
was  also  compared  with  the  personnel  characteristics  of  the  RDs  aboard 
the  same  ship  in  the  same  quarter.  Each  set  of  values  for  the  equipment 
and  personnel  data  for  a  given  ship  in  one  of  the  selected  3-month  periods 
is  a  sample  point  for  use  in  the  multivariate  correlational  analyses.  The 
methodology  used  is  described  in  Appendix  E. 


TABLE  17 

SHIPS  CHOSEN  FOR  THE  ANALYSIS 


AN/S  PS- 

■40  Radar 

AN/SOS- 

23B  Sonar 

DD  692 

DD  785 

CLG 

7 

DD  693 

DD  787 

DD 

836 

DD  694 

DD  788 

DD 

851 

DD  698 

DD  823 

DD 

852 

DD  699 

DD  843 

DD 

870 

DD  703 

DD  846 

DD 

888 

DD  755 

DD  862 

DD  759 

DD  869 

J 

DD  761 

DD  871 
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4.56  The  criterion  variables  used  in  the  analysis  and  applied  to  each 
of  the  equipments  are: 

a.  Mean  number  of  man-hours  to  repair 

b.  Mean  number  of  man-hours  to  repair  by  type  of  failure 

c.  Number  of  repairs  (malfunctions) 

d.  Number  of  repairs  by  type  of  failure. 

4.57  Criterion  variable  c  reflects  a  count  of  the  total  number  of  repairs 
that  occurred  or.  each  ship  in  each  quarter.  Variable  d  classifies  these 
into  one  of  94  different  types  of  malfunctions  as  coded  by  the  3-M  system. 
Criterion  variables  a  and  b  are  self-explanatory. 


4.58  The  following  predictor  variables  were  used  in  the  analysis  and 
applied  to  each  of  the  equipments: 


1. 

2. 

• 

4. 

5. 

6. 

7. 

8. 
9. 

10. 

11. 


Average  age 
Average  pay  grade  * 

Average  number  of  months  since  ADBD 
Average  number  of  months  until  EAOS 
Years  of  formal  education 

Percent  of  personnel  allowance  actually  aboard 
Percent  of  personnel  with  NEC  pertinent  to  equipment  ** 
Average  weeks  of  training  time  in  specialized  "C"  schools 
Average  number  of  months  since  ADBD  for  the  highest  ranked 
man 

Years  of  formal  education  of  the  highest  ranked  man  *** 
Average  number  of  months  remaining  to  EAOS  for  all  men 
serving  their  first  enlistment.  **** 


*  In  calculating  variables  2  and  5,  a  numerical  pay  grade  was  assigned 
to  pay  grades  and  years  of  education. 

**  1 6/ 
Reference  was  made  to  the  Manual  of  Navy  Enlisted  Classifications— / 

to  determine  NECs  applicable  to  the  equipments;  these  were: 

Equipment  Applicable  NECs 

AN/SPS-40  1514 

AN/SQS-23B  0407,  0404,  0486,  0487,  0488, 

0494,  0496 

In  calculating  variable  10,  a  numerical  code  was  assigned  to  year 
of  education. 

Men  serving  their  first  enlistment  are  defined  as  those  whose  ADBD 
subtracted  from  their  EAOS  resulted  in  a  number  less  than  or  equal 
to  4  years . 
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4.59  All  of  the  predictor  variables  were  applied  to  all  men  in  each  of 
the  RD,  ET,  and  ST  ratings  who  were  aboard  a  given  ship  during  a 
given  quarter  selected  for  the  sample. 

4.60  The  alternate  hypotheses  based  on  these  variables  are: 

a.  Average  age  is  negatively  correlated  with  average  time 
to  repair  and  number  of  malfunctions 

b.  Average  pay  grade  is  negatively  correlated  with  average 
time  to  repair  and  number  of  malfunctions 

c.  Average  number  of  months  for  all  personnel  since  ADBD 
is  negatively  correlated  with  average  time  to  repair  and 
number  of  malfunctions 

d.  Average  number  of  months  for  all  personnel  until  EAOS 
is  negatively  correlated  with  average  time  to  repair  and 
number  of  malfunctions 

e.  Average  formal  education  is  negatively  correlated  with 
average  time  to  repair  and  number  of  malfunctions 

f.  Percent  of  personnel  allowance  actually  aboard  is 
positively  correlated  with  mean  time  to  repair  and 
negatively  correlated  with  number  of  malfunctions 

g.  Percent  of  personnel  with  NEC  pertinent  to  equipment 
is  negatively  correlated  with  mean  time  to  repair  and 
number  of  malfunctions 

h.  Average  training  time  in  specialized  "C"  schools  is 
negatively  correlated  with  mean  time  to  repair  and 
number  of  malfunctions 

i.  Time  since  ADBD  for  the  highest  ranked  man  is  nega¬ 
tively  correlated  with  mean  time  to  repair  and  number 
of  malfunctions 

j.  Formal  education  of  the  highest  ranked  man  is  negatively 
correlated  with  mean  time  to  repair  and  number  of  mal¬ 
functions 

k.  Average  number  of  months  remaining  until  EAOS  for  all 
men  serving  their  first  enlistment  is  negatively  correlated 
with  mean  time  to  repair  and  number  of  malfunctions. 


69 


4.61  The  null  hypothesis  associated  with  each  of  these  hypotheses  is 
that  the  personnel  (or  predictor)  characteristic  is  not  related  to  the  equip¬ 
ment  (or  criterion)  variable . 

4.62  In  addition,  multiple  correlations  were  run  using  a  combination 
of  personnel  characteristics  in  order  to  find  significant  relationships  with 
each  of  the  equipment  parameters. 

4.63  Initial  correlation  analyses  between  equipment  repairs  and  mal¬ 
functions  in  each  quarter  and  characteristics  of  the  personnel  in  the  same  - 
quarter  showed  no  significant  multiple  correlations.  These  results  point 
out  that  the  characteristics  of  the  people  working  on  the  equipment  in 

any  quarter  have  little  effect  on  the  number  of  malfunctions  in  that  same 
quarter. 

4.64  However,  it  also  seemed  plausible  that  their  errors  may  not  result 
immediately  in  malfunctions  or  lengthy  repairs  but  may  cause  failures  in 
succeeding  quarters.  Thus,  analyses  have  been  made  that  correlate 
personnel  data  in  one  quarter  with  the  equipment  failure  parameters  3  months 
or  6  months  later. 

4.  65  Also  considered  was  the  possibility  that  changes  from  quarter  to 
quarter  in  personnel  and  equipment  failure  data  would  be  more  sensitive 
to  significant  relationships  than  the  actual  data.  Inspection  of  the  personnel 
showed  a  fairly  low  turnover  from  one  quarter  to  the  next.  Using  the  changes 
in  personnel  and  failure  data  also  eliminates  any  possible  trends  which  may 
occur  on  a  given  ship  over  a  series  of  time  periods  causing  spuriously  high 
correlations.  Therefore,  a  limited  number  of  analyses  was  done  using 
changes  rather  than  actual  data.  The  number  was  limited  because  of  the 
resulting  decreased  sample  size. 

4.66  Table  18  shows  all  of  the  analyses  run  with  the  sample  size  for 
each.  Correlations  were  run  using  the  "no  output"  malfunction  of  the 
AN/SPS-40  and  the  "burned  out"  malfunction  of  the  AN/SQS-23B  as  the 
criterion  variables.  These  two  malfunctions  were  chosen  because  they 
occurred  more  frequently  than  any  other  on  the  respective  equipments. 

Results 

4.67  Summary.  Five  multiple  correlations  are  significantly  different  from 
zero  at  the  95  percent  confidence  level.  All  significant  multiple  correlations 
relate  personnel  characteristics  from  one  quarter  to  equipment  failures  or 
repairs  occurring  in  the  next  quarter. 

4.68  There  were  also  9  gross  correlations  involving  6  personnel  param¬ 
eters  which  were  significantly  different  from  zero. 
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TABLE  18 

CORRELATION  CASES  RUN 


Description 

Case  Number 

Sample  Size 

Radarman 

Actual  data 

All  malfunctions  reported 

No  lag 

1 

50 

Lag  3  months 

2 

33 

Lag  6  months 

3 

16 

"No  output"  malfunction 

No  lag 

4 

24 

Lag  3  months 

5 

18 

Change  data 

All  malfunctions  reported 

No  lag 

6 

31 

Lag  3  months 

7 

15 

Electronic  technicians 
Actual  data 

All  malfunctions  reported 


No  lag  8  50 

Lag  3  months  9  33 

Lag  6  months  10  16 

"No  output"  ma’functions 

No  lag  11  24 

Lag  3  months  12  18 

Change  data 

All  malfunctions  reported 

No  lag  13  31 

Lag  3  months  14  15 


Sonar  technicians 
Actual  data 

All  malfunctions  reported 

No  lag  15  23 

"Burned  out"  malfunction 

No  lag  '  16  16 

Change  data 

All  malfunctions  reported 
No  lag 
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4.69  Multiple  Correlations.  There  are  five  significant  multiple 
correlations. 


a.  Changes  from  quarter  tc  quarter  in  the  average 
time  since  ADBD  and  average  formal  education 
of  the  radarmen  are  significantly  related  to  the 
quarterly  changes  in  the  total  number  of  AN/SPS-40 
malfunctions.  The  actual  correlation  was  0.693 
and  one  can  be  95  percent  confident  that  the  true 
correlation  is  greater  than  0.23.* 

b.  The  percentage  of  electronics  technicians  with  an 
NEC  pertinent  to  the  AN/SPS-40  radar  and  with  the 
average  training  time  in  specialized  "C"  schools 
is  significantly  related  to  the  total  number  of  mal¬ 
functions.  The  actual  correlation  was  0.482  and, 
with  95  percent  confidence,  it  can  be  said  that  the 
true  value  of  the  correlation  is  greater  than  0.13.* 

c.  Changes  from  quarter  to  quarter  in  the  average  time 
since  ADBD  and  the  percentage  of  radarmen  aboard 
to  the  radarman  allowance  are  significantly  related 
to  changes  from  quarter  to  quarter  in  the  mean  time 
to  repair  all  malfunctions  of  the  AN/SPS-40.  The 
actual  correlation  was  0.809  and  there  is  again 

95  percent  confidence  that  the  true  correlation  is 
greater  than  0.53.* 

d.  The  average  time  until  EAOS  for  both  electronics  tech¬ 
nicians  serving  their  first  enlistment  and  all  ETs 
combined  and  the  percentage  of  electronics  technicians 
with  an  NEC  pertinent  to  the  AN/SPS-40  are  signifi¬ 
cantly  related  to  the  mean  time  to  repair  all  malfunctions. 
T  ’  actual  correlation  was  0.616  and  statistical  theory 
indicates  95  percent  confidence  that  the  actual  correlation 
is  greater  than  0.34, 

e.  Changes  from  quarter  to  quarter  in  the  average  pay  grade 
and  the  percentage  of  electronic  technicians  with  an  NEC 
pertinent  to  the  AN/SPS-40  are  significantly  related 

to  the  quarterly  changes  in  the  mean  time  to  repair  all 
malfunctions  of  the  AN/SPS-40.  The  actual  correlation 
was  0.646  and  again,  it  is  95  percent  certain  that  the 
true  correlation  is  greater  than  0.15.* 


Approximate  value  obtained  through  interpolation  in  graphs. 
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4.70  There  are  several  interesting  observations  that  can  be  made 
concerning  these  findings.  First,  each  significant  relationship  involved 
characteristics  of  personnel  in  the  quarter  prior  to  the  equipment  failure 
or  repair. 


,  Time  Lag  to  Failure 

Significant 

Correlations 

r 

Nonsignificant  ; 
Correlations  j 

Total 

Cases  with  no  lag 

0 

18 

18 

Cases  with  3-month  lag 

5 

7 

12 

Cases  with  6-month  lag 

» 

4 

.  .  _  i. 

i 

j* 

i _ 

4.71  This  observation  suggests  that  there  may  be  a  lag  between  the  time 
a  person  causes  a  failure  in  some  manner  and  the  time  when  the  failure 
actually  occurs  and  is  repaired. 

4.72  The  second  observation  that  can  be  made  is  that  all  significant 
multiple  correlations  involved  the  AN/SPS-40  radar.  None  occurred  with 
the  AN/SQS-23B  sonar,  which  may  be  attributable  to  the  reduced  sample 
size . 

4.73  Third,  none  of  the  multiple  correlation  analyses  involving  specific 
malfunctions  was  significant.  All  significant  analyses  used  the  total  number 
of  all  types  of  malfunctions.  This  situation  may  be  due  to  the  generally  small 
sample  sizes  available  for  specific  malfunctions. 

4.74  Complete  listings  of  all  maximum  multiple  correlation  coefficients 
are  found  in  Tables  19  and  20.  An  ORI  computer  program  furnished  correlations 
for  all  possible  combinations  of  personnel  characteristics  with  each  of  the 
equipment  parameters.  For  each  multiple  correlation  analysis  using  a  dif¬ 
ferent  number  of  variables,  only  that  analysis  showing  the  highest  correla- 
lation  was  selected  for  exhibit.  In  all  cases,  the  correlation  coefficients  were 
adjusted  (decreased)  for  degrees  of  freedom  and  sample  size  according  to  the 
formula  presented  in  Appendix  E.  This  adjustment  accounts  for  the  fact  that  some 
have  a  value  of  zero.  Beside  each  coefficient  are  the  coded  numbers  for 

the  personnel  characteristics  entered  into  the  multiple  correlation  analysis.* 
Blank  spaces  in  the  table  result  because  not  all  personnel  characteristics 
were  used  in  each  correlation  analysis  either  because  of  lack  of  data  or 
as  a  result  of  the  judgment  that  there  was  no  logical  reason  why  it  should 
be  related  with  the  criterion. 


The  numbers  correspond  to  the  predictor  variables  numbered  in 
paragraph  4.58. 
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TABLE  19 

maximum  multiple  correlation  coefficients-**/ (E)  with  number  of  malfunctions 
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4.75  Some  measure  of  the  relative  importance  of  the  personnel 
characteristics  which  make  up  the  significant  multiple  correlations  is 
the  /£  coefficient  shown  in  Table  21 .  The  larger  the  coefficient,  the 
closer  It  relates  to  and  Influences  the  equipment  variable. 

4.76  Gross  Correlations.  Table  22  shows  that  average  formal 
education,  the  percentage  of  personnel  allowance  actually  aboard, 
and  the  percentage  of  the  crew  with  an  NEC  pertinent  to  the  equipment 
had  the  greatest  number  of  significant  results.  These  are  the  correlations 
which  are  significantly  different  from  zero. 

4.77  Nine  gross  correlations  were  significantly  different  from  zero. 
Tables  23  and  24  present  all  correlations  calculated  for  each  of  the 
personnel  characteristics. 

4.78  Tables  23  and  24  also  show  all  gross  correlation  coefficients 
after  adjustment  for  sample  size.  Again,  zero  values  are  caused  by  the 
adjustment.  These  coefficients  show  the  relationship  of  each  personnel 
characteristic  with  the  equipment  variables. 

4.79  Limitations .  The  prime  limitation  in  the  study  is  the  nonrandom 
selection  of  sh.ps  for  the  sample.  The  nonrandom  selection  was  necessary 
because  of  the  differing  quality  of  the  3-M  reporting  by  ships.  Ships  were 
selected  which  had  the  most  accurate  data.  As  a  result,  significant  findings 
related  to  the  sample  ships  may  not  necessarily  apply  to  personnel  and 
equipment  on  ships  not  included  in  the  sample.  A  study  involving  a  signifi¬ 
cantly  larger  randomly  selected  data  base  should  be  performed  to  verify 

the  findings. 

4.80  Those  personnel  characteristics  which  were  found  to  be  highly 
related  to  the  AN/SPS-40  or  AN/SQS-23B  failure  frequency  and  repair 
times  may  not  be  the  same  characteristics  which  are  correlated  to  failure 
and  maintenance  parameters  for  other  equipments. 

4 ‘81  Conclusions.  This  research  demonstrates  the  ability  of  multiple 
correlation  analysis  to  pinpoint  personnel  characteristics  which  influence 
equipment  failures  and  repair  times.  Its  use  does  not  require  additional 
data  collection  but  uses  existing  Navy  data  systems.  The  computations 
are  extremely  laborious  and  require  ADP.  The  availability  of  the  ORi 
multivariate  analysis  computer  program  greatly  facilitates  the  use  of  a 
computer  in  the  application  of  this  technique. 
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TABLE  22 

SIGNIFICANT  GROSS  CORRELATIONS 


Personnel 

Characteristics 

No.  of 
Significant 
Correlations 
With  No.  of 
Malfunctions 

No.  of 
Significant 
Correlations 
With  Mean 

Time  to  Repair 

Total 

Significant 

Correlations 

Average  age 

Average  pay  grade 

1 

1 

Average  time  since 

ADBD 

i 

1 

Average  time  until 

FA  OS 

1 

1 

Average  formal 
education 

2 

2 

Percentage  personnel 
allowance  actually  aboard 

1 

1 

2 

Percentage  personnel 
with  pertinent  NEC 

1 

1 

2 

Average  training  time 

Time  since  ADBD  for 
highest  ranked  man 

Formal  education  for 
highest  ranked  man 

Tl;  until  EAOS  for 
men  on  first  enlistment 
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TABLE  24 

GROSS  CORRELATION  COEFFICIENTS-V(R)  FOR  ALL  CASES  WITH  MEAN  TIME  TO  REPAIR 

I  Personnel  Chorecterlatlci  V 


o 


o  o  o 


o  o 


o  o 
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original  list  In  paragraph 


V.  SUMMARY  OF  RESULTS  AND  RECOMMENDATIONS 


RESEARCH  RESULTS 

5. 1  The  research  effort  focused  on  two  major  areas:  a  survey  and 
analysis  of  existing  failure  reporting  systems,  and  the  investigation  of 
alternative  indirect  approaches  to  determine  human  performance  and  to 
quantify  the  human  reliability  contribution  to  weapon  system  effec¬ 
tiveness  . 

5.2  It  was  found  that  existing  failure  reporting  systems  do  not  yield 
meaningful  data  on  human-initiated  malfunctions.  Most  systems  have  no 
provisions  for  reporting  information  on  human  performance  as  related  to 
failures.  Many  of  the  reporting  systems  surveyed  were  in  various  stages 
of  being  phased  out  of  use.  In  most  cases,  a  strong  reluctance  to  report 
all  failures,  particularly  human  errors,  was  noted.  What  information  was 
found  to  exist  on  human  errors  was  very  general  in  nature  and  was  not 
being  used  for  human  failure  analysis.  Indeed,  meaningful  analysis  of 
the  reported  data  did  not  appear  feasible.  Some  provisions  were  being 
made,  however,  for  inclusion  of  human-initiated  failure  data  in  the  3-M 
system  as  described  in  paragraph  3.41.  These  provisions  warrant  further 
investigation  for  possible  improvements  that  will  make  them  useful  in  human 
reliability  analysis. 

5.3  In  attempting  to  develop  an  indirect  approach  to  human  reliability 
analysis,  two  techniques  were  investigated,  both  of  which  rely  on  equip¬ 
ment  failure  reporting  rather  than  human  error  reporting.  One  technique  is 
ERUPT.  This  approach,  by  grouping  the  components  of  a  weapon  system 
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into  elementary  reliability  units,  provides  a  means  of  inferring  two  human 
performance  parameters  from  available  equipment  reliability  and  mainte¬ 
nance  data.  The  second  approach  relates  certain  personnel  characteris¬ 
tics  of  individuals  operating  and  maintaining  the  equipment  to  the  number 
of  failures  and  equipment  repair  times  by  the  application  of  multivariate 
correlation  analysis  techniques. 

5.4  In  the  course  of  this  study,  the  feasibility  and  applicability  of 
these  two  approaches  were  determined  and  mathematical  models  and  equa¬ 
tions  were  developed  for  their  applications.  A  pilot  test  on  a  limited  data 
base  was  also  conducted  during  the  development  of  the  two  techniques. 

RECOMMENDATIONS 

5 . 5  The  conduct  of  the  study  and  the  conclusions  point  to  a  number 
of  recommendations  which  logically  fail  into  two  categories,  those  for 
immediate  action  and  those  for  further  research  and  study.  This  section 
delineates  these  recommendations . 

Immediate  Action 

5.6  Recommendation  1.  One  of  the  major  problems  associated  with 
failure  reporting  systems  is  their  use  for  disciplinary  purposes.  It  is  strongly 
recommended  that  BuPers  take  Immediate  action  to  Initiate  an  educational 
program  clarifying  the  basic  purposes  of  these  reporting  systems.  This  edu¬ 
cational  program  should  emphasize  the  need  for  accurate  data  on  failures . 

The  program  should  further  emphasize  that  the  data  are  required  for  the  enhance¬ 
ment  of  technical  analyses  and  further  study,  rather  than  for  the  evaluation  of 
personnel  and  promotion  decisions. 

5 . 7  Recommendation  2 .  The  Shipboard  Maintenance  Action  Form 
(Figure  15)  of  the  3-M  system  provides  for  the  name,  rating,  and 
grade  of  the  person  who  performs  the  maintenance  action  being  reported. 

This  information  would  be  extremely  useful  in  conducting  analyses  such  as 
the  ones  outlined  in  Section  IV.  These  data  are  presently  not  being  keypunched 
and,  consequently,  are  not  available  in  routine  or  special  3-M  reports.  It  is 
recommended  that  the  next  revision  to  the  3-M  manual  incorporate  the  instruc¬ 
tions  for  keypunching  these  elements. 


Further  Research  and/or  Study  Programs 

5.8  Recommendation  1.  The  ERUPT  approach  appears  to  be  feasible 
and  is  believed  to  be  one  of  the  most  promising  techniques  thus  far  developed 
for  human  reliability  analysis.  It  is  therefore  strongly  recommended  that 
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this  approach  be  further  developed  by  extending  its  application  to 
classes  of  weapon  systems  other  than  the  one  employed  during  its 
formulation.  Its  usage  should  also  be  Investigated  in  system 
effectiveness  analyses  during  various  phases  of  the  weapon  system 
development  cycle. 

5.9  The  following  specific  tasks  are  recommended  as  part  of  this 
subsequent  study  program: 

a.  Investigate  the  application  of  ERUPT  to  various 
classes  of  weapon  systems  as  it  applies  to  system 
readiness  reliability. 

b.  Investigate  the  application  of  ERUPT  to  other  classes 
of  systems  where  human  reliability  parameters  are  an 
integral  part  of  the  system  effectiveness  considerations 
in  all  phases  of  the  system  life  cycle.  This  task  will 
Include  the  formulation  of  human  performance  parameters 
other  than  a  and  0  delineated  in  the  description  of  the 
ERUPT  approach  in  Section  IV.  The  effort  will  involve 
the  mathematical  formulation  of  an  overall  system 
effectiveness  model  which  will  include  these  human 
performance  parameters  along  with  additional  equipment 
parameters . 

c.  Calculate  a,  8,  and  other  human  reliability  parameters 
for  ERUs  of  various  classes  of  systems,  based  on  actual 
equipment  reliability  and  system  operational  and  mainten¬ 
ance  data.  Analyze  similar  ERUs  to  develop  common 
human  reliability  parameters.  Classify  these  ERUs  based 
on  ranges  of  the  human  reliability  parameters. 

d.  Develop  relationships  between  various  human  reliability 
parameters  and  personnel  characteristics.  Develop 
sensitivity  curves  for  sets  of  personnel  characteristics 
vs  human  reliability  parameters  for  which  a  significant 
relationship  has  been  established. 

5.10  Based  on  the  results  of  these  tasks,  meaningful  human  reliability 
measures  can  be  formulated  which  will  be  of  significant  value  to  BuPers 
representatives  participating  In  weapon  system  effectiveness  analysis. 
These  measures  can  be  applied  to  the  analysis  of  existing  systems  as  well 
as  of  those  in  various  stages  of  early  development.  Further,  these  measures 
will  facilitate  the  evaluation  of  such  system  criteria  as  packaging  concepts, 
maintenance  philosophy,  and  operating  procedures  in  terms  of  manpower 
and  personnel  requirements . 
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5.11  Recommendation  2.  The  application  of  multivariate  correlation 
analysis  to  equipment  failure  and  personnel  characteristics  data  showed 
some  significant  relationships  based  on  a  limited  statistical  sample.  It 
is  recommended  that  the  validation  of  the  results  of  the  study  and  the 
extension  of  the  techniques  to  other  equipment  be  undertaken.  This  effort 
will  require  the  following  tasks: 

a.  Validation  of  the  significant  findings  of  the  research 
conducted  in  this  study,  which  will  require: 

1 .  Additional  personnel  data  for  RD,  ST,  and 
ET  ratings 

2 .  Additional  sonar  and  radar  equipment 
failure  data  for  a  larger  random  sample  of 
ships 

b.  Application  of  the  technique  to  other  classes  of 
equipment  that  are  amenable  to  use  of  this 
technique 

c.  Formulation  of  large  numbers  of  hypotheses 
concerning  personnel  and  equipment  fa  'are 
parameters  through  contacts  with  navai 
personnel  who  are  familiar  with  personrel 
requirements  of  those  who  operate  and  maintain 
the  equipments 

d.  Determination  of  the  availability  of  personnel 
data  from  Pers  N  needed  to  test  the  hypotheses 
developed 

e.  Finally,  a  multivariate  correlational  analysis 
to  reveal  which  personnel  characteristics  are 
highly  related  to  changes  in  each  equipment's 
failure  parameters.  Analysis  would  determine 
whether  the  same  personnel  parameters  were  re¬ 
lated  tc  similar  types  of  equipment. 

A  matrix  format  might  be  feasible  for  presentation  and  summarization  of 
the  correlations  found  for  each  of  the  personnel  parameters  with  the 
different  equipment  classes. 

5.12  Recommendation  3.  Tne  forms  that  are  used  to  report  failures 
constitute  one  of  the  major  problems  associated  with  existing  failure 
reporting  systems.  There  is  no  way  to  isolate  human-initiated  failures 
from  equipment  breakdox  n.  It  is  recommended  that  a  study  be  conducted 
to  design  a  failure  reporting  form  which  would  assure,  with  a  high  degree 
of  confidence,  the  identification  of  the  true  causes  of  failures. 
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The  study  would  Include  the  investigation  of  the  application  of  some 
form  of  a  failure  analysis  tree  which  would  take  the  reporter  through 
a  series  of  "yes"  or  "nc"  decisions  to  arrive  at  the  final  conclusion 
as  to  the  cause  of  failure.  The  study  would  require  the  determination 
of  information  needs  for  equipment  analyses  as  well  as  human  reliability 
analyses  to  assure  the  Inclusion  on  the  form  of  only  those  elements  that 
are  absolutely  required  in  support  of  the  analyses.  It  is  further  recommended 
that  various  alternative  reporting  concepts  be  pilot  tested  under  actual 
operating  and  maintenance  conditions. 
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APPENDIX  A 


DERIVATION  OF  EQUATIONS  FOR  HUMAN  RELIABILITY  PARAMETER 
ESTIMATES  AND  READINESS  RELIABILITY 


A. I 


The  following  terms  must  be  defined: 

=  probability  that  failure  of  the  ith  ERU , 
if  it  exists,  is  detected  and  repaired 
during  the  first  maintenance  following 
the  failure. 


Si  = 


v 


ij 


t. 

1 


l-G.(x)  = 

1 


probability  that  maintenance  does  not  induce 
failure  in  the  i*h  ERU  given  that  the  ERU 
is  in  nonfailed  condition  at  the  time  main¬ 
tenance  is  initiated. 

probability  that  i**1  ERU,  which  is  in 
nonfailed  condition  at  the  time  the  j**1 
exercise  or  test  is  initiated,  survives  the 
exercise  or  test. 

storage  time  between  exercises  or  tests  of  i**1 
ERU. 

probability  that  i**1  ERU  survives  a  storage  time  x 
given  that  it  was  in  "new"  condition  at  beginning 
of  storage,  i.e.,  at  zero  storage  time.  This  is 
determined  from  a  theoretical  distribution  estimated 
under  laboratory  or  test  conditions . 
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NK 


number  of  different  integral  values  of  k  in 
the  sample  of  corrective  maintenances. 


ki 


Ni 


Ti 


yi 


number  of  maintenances  before  repair/replacement 
since  beginning  of  storage  or  since  last  repair 
of  the  ERU  if  previous  corrective  maintenance 
has  been  done  (including  maintenance  when  last 
repair/replacement  was  done).  A  sample  of  kj's 
is  required  from  actual  experience  to  derive 
maximum  likelihood  estimates  of  a  and  0. 

(i  ~  1,  .  .  .  NK). 

frequency  of  the  i**1  value  of  k^  in  the  sample  of 
corrective  maintenances  (i  =  1 ,  2,  ...NK).  Size 
of  sample  is  equal  to 

NK 

E  Nt. 

i=l 


storage  time  of  ith  ERU  between  time  when  last 
exercised  and  maintained  to  time  of  operating 
mission. 

number  of  exercises  or  tests  of  ERU  before 
operating  mission, 

probability  that  the  ith  ERU  survives  the 
exercise  or  test  and  was  last  repaired  during 
the  (i^-j)  maintenance  period. 


ESTIMATION  OF  HUMAN  RELIABILITY  MAINTENANCE  PARAMETERS 


A. 2  Assume  -  ffi  for  all  values  of  j ;  Pj,,(r)  -  probability  that 
corrective  maintenance  (which  assumes  complete  renewal)  takes  place  on 
the  kth  maintenance;  and  Px(f)  =  probability  that  a  failure  occurs  before  the 
xth  maintenance  but  not  before  the  (x-l)tk  maintenance;  i.e.,  the  failure 
could  have  occurred  during  the  (x-l)th  maintenance,  during  storage  between 
the  (x~l)th  and  xth  maintenances,  or  during  the  xth  test. 

p  (f)  =  i  -  fl  -  G  (t)  1  ft.  (A»*) 
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px(f)  =  jSX”2  Qi-G  (x-1)  tjff5'"1  -(P~l  (Xt)]  1TX  for  x  *  2.  (A. 2) 

A.  3  These  formulations  are  obvious  after  inspection  of  input  data  and 
reexamination  of  the  life  cycle  of  the  ERU  as  depicted  in  Figure  18. 

A. 4  Assuming  a  failure  before  the  first  maintenance,  the  probability 
that  the  corrective  maintenance  will  take  place  on  the  fourth  maintenance 
is  simply: 

Pj(f)  •  (1-a)3  a  =  l  - [i-G(t)]  n  (i-a3)a.  (A. 3) 

A. 5  More  generally,  the  probability  that  corrective  maintenance 
occurs  on  the  k**1  maintenance  for  a  failure  occurring  before  the  main¬ 
tenance  is: 

px(f)  •  (l-a)k"x  a. 


k 

Now,  pk(r)  =  V  Px(f)  .  (i-a)k_x  a. 


(A. 4) 


Expanding ,  ^ 

Pk(r)  =  E  |^X"2  [l-G  (x-1)  t]  -j3X_1  [l-G  (xt)]  *X  j(l-,y)  k_x  <* 

+  jl-  [l-G(t)]  a.  {A*5) 

A. 6  This  expression  gives  the  probability  that  corrective  maintenance 
occurred  on  the  k*^1  maintenance  regardless  of  when  the  failure  occurred. 

A. 7  Assume  now  that  a  sample  of  ERU  corrective  maintenances  shows 
that  they  have  occurred  on  the  first,  third,  third,  second,  and  first  preventive 
maintenances.  Thus,  under  the  definitions, 

NK  =  3  Kj  =  1  Ni  =  2 

K2  =  2  N2  =  1 

K3  =  3  N3  =  2. 
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A.  8  The  probability  that  this  series  of  corrective  maintenance  has 

occurred  equals 


3 

n 

i=l 

More  generally,  the  probability  is 


(r). 


A. 9  The  set  of  (at/3)  desired  is  the  set  that  maximizes  this  probability. 

Therefore,  the  desired  ( a,/3 )  equals 


A.  10  A  computer  program  to  facilitate  calculation  of  (a,0)  is  given 

in  Appendix  B. 


READINESS  RELIABILITY  EQUATIONS 

A.  11  It  can  be  shown  that  the  readiness  reliability  of  the  ith  ERU  is 

given  by* 

*  Equation  (A. 8)  for  readiness  reliability  of  the  ERU  can  be  derived  by  induc¬ 
tion.  The  boundary  condition  Pi ( 1 )  =  L 1  -G^ti)]  ffi(  1 )  is  based  on  the  assump¬ 
tion  that  the  ERU  is  in  operable  condition  with  probability  unity  immediately 
following  manufacture . 
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Ji 

W  V  V  '  ai[1')fipj<‘'i,I!-Gl'Ti). 

f  r  i-G  o  t  +ri) 

+  8  V"  ! - i - I - r  (y) 

1  £l[  '-G.tJV  ^ 

where  the  recursion  equations  necessary  to  the  formulation  are 

p1)<vl)][1-Gi«v]’,(,'i»- 

Pj(l/i)  “  pi  i-Gjo-Dt^  pi>i(V1)V,V 


p^i)  =  ^i  i-G^-DtJ  pi,i/i.1vVj;"i'V' 

A.  12  For  the  case  where  the  distribution  of  storage  time  to  failure 
of  the  ERU  is  exponential  and  the  probability  of  surviving  the 
exercise  ffjj  =  for  all  ),  it  can  be  shown  that  Equation  (A. 8)  takes  on 
the  simpler  form 

-\\T,  I  r  .  tI'  -X'l/t. 


(u-l)  ir  ( v). 


-X.r,  r  'll'.  -A«Pt 

Rl  =  1  [’i  faj  1 


(A.  8) 


(A. 9) 


From  inspection  of  Equation  (A. 9)  it  can  be  seen  that,as  approaches 
8^,  the  readiness  reliability  of  an  ERU  having  an  exponential  distribution  of 
storage  time  to  failure  will  approach  which  is  independent  of 

Up  the  number  of  maintenance  cycles  prior  to  operating  mission.  For  the 
exponential  case,  note  also  tnat  as  pj  becomes  large,  the  steady-state 
readiness  reliability  approaches 


Ri  - 


al 

i 


-x;ti 


■-ifVi) 


1"^  1*1 


(A. 10) 


System  Readiness  Reliability 

A.  13  Taking  the  product  of  ERU  readiness  reliabilities  defined  by 
Equation  (  A. 8)  for  ERUs  required  to  function  for  a  defined  mission  tactic 
yields  the  readiness  reliability  Rr.  In  mathematical  form,  this  is  given 
by 


R 


R 


R, 


(A. 11) 
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APPENDIX  B 

FLOW  DIAGRAM  AND  COMPUTER  PROGRAM  FOR 
ESTIMATING  HUMAN  RELIABILITY  PARAMETERS 


B.  1  The  variables  and  format  used  for  the  input  data  required  for 
the  program  which  estimates  human  reliability  factors  are  shown  in 
Table  B.  I. 


TABLE  B.l 

INPUT  DESCRIPTION 


Card 

Card  Column 

Description 

1 

1-10 

NK,  total  number  of  kj's  or  Nj's 

11-20 

probability  of  surviving  test 
or  exercise 

2 

1-10 

AINI,  initial  value  of  alpha 

11-20 

AINC,  incremental  value  of  alpha 

21-30 

AFIN,  terminal  value  of  alpha 

31-40 

BINI,  initial  value  of  beta 

41-50 

BINC,  incremental  value  of  beta 

51-60 

BFIN,  terminal  value  of  beta 

B-l 


TABLE  B.  1  (Cont) 


Card 

Card  Column 

Description 

3* 

1-10 

k^,  number  of  maintenances  before 
repair/replacement  since  begin¬ 
ning  of  storage 

11-20 

N^,  frequency  of  the  i**1  value  of  kj 

4 

1-5 

G  (It)  G(xt)  table  values 

6-10 

G  (2t) 

11-15 

• 

• 

G(3t) 

• 

• 

• 

46-50 

• 

• 

G(  10) 

♦Card  3  repeated  NK  times  for  each  k^,  Nj. 

B.2  The  program  will  print  all  input  data  in  Table  B.  1  as  well  as  the 
data  calculated  by  the  program  and  shown  in  Table  B.2. 


TABLE  B.2 


OUTPUT  DESCRIPTION 


FORTRAN  Name 

Description 

ALPHA 

Probability  of  detecting  a  malfunction  during 
maintenance  (a) 

BETA 

Probability  of  not  inducing  a  malfunction  during 
maintenance  (/3) 

PROD 

Probability  that  this  sample  of  corrective  main¬ 
tenances  could  have  occurred  given  this  set 
of  alpha  and  beta  values 

PMAX 

Maximum  probability  that  this  sample  of  correc¬ 
tive  maintenance  could  have  occurred. 

A  L 

Value  of  aJpha  at  which  maximum  (PMAX)  occurs 

BL 

Value  of  beta  at  which  maximum  (PMAX)  occurs 
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FIGURE  B.l  (Cont) 
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FIGURE  B.  I  (Cont) 
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FIGURE  B.  1  (Cont) 
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FIGURE  B .  1  (Cont) 
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3200  FORTRAN 


(2.2) 


67/03/01 


PROGRAM  IIMMS 

U 1  MENS  1 ON  MlO)  »N(10)  *6XT(10> 

1  HcAfXbO*  I00>  NMPROH 

GO  TO  «F_UFCKF'  (60) 

2M  Ht AIH60* 101 ) A (Nl *AlNC«AF IN*HIN1 *HlNC*4FlN 
Ht  AIJ  (60*  102)  (MI).N(l).  1=1 *NK) 

KtALUbO.  103)  (GXT  (  I)  ♦  1  =  1*10) 

"HlTt(61*i04)PROB»NK 

"HI  lfc(61*  1 10)  (Ml)*  N  ( I )  *  1  =  1. NK) 

"Hi Tt (61 » 105) 

"Hi TE (61  *  1 1 1 )  (I*  6X1(1)*  1  =  1*10) 

"Hilt (61 ♦ 106) AlNl *AFIN*AINC 
"HI TE (61»112)HINI .HFIN.bINC 

"Hi  1 1  <  6 1 » 1 0 7 ) 

AF  I  N=  AFlN+.OOl 
BF I N=HF IN+.001 
P1=PR0H 

CGXT=l.-( (l.-GXl (1) )«PI> 

PM A  X  =  —  l . 

ALPHA  =  AIM 
33  CALPH=l. -ALPHA 
BE I A=h INI 
IB  PHOI)=l  . 

UO  2  1=1 *NK 

M=K<  I) 

IF  (M-l)4*4*b 
b  PALPh=CALHH**(M-l) 

GO  10  6 
4  PALPh=l. 

6  LSTN I =CGA I *PALPH*ALPHA 
P6UM=0. 

UO  11  IX  =  2*M 
IF  ( IX-2J9.9* 10 

9  bFuNC=l.-CGX(-«tTA*(l.-6Xr(2) )*PI*P1 
IF  IK1-1X) 45* 13. 15 
4b  Ml  X=  1  Abb  ( K  1  —  I X ) 

AKIJNC-  (  l  ./CALPH*«KI1X  )  *ALPHA 
GO  TO  2b 
13  AF  UNC  =  ALPHA 
GO  TO  /b 

15  AF  UNC=C  ALPFIA*  ( rv  I  —  I  X  )  «aLPHA 
go  In  26 

1U  bFUNC= (Ht 1  A** ( IX-/) ) * ( l ,-GXT ( ix-l ) ) « (Pl«» ( IX-1 ) ) - (bET A** ( IX-1 ) ) 
l*(I»  —  bXl  (IX) )*<PI**1A> 

IF  ( K I - l X ) 46  *24  »  2b 
46  M IX=1 AdS(Kl-lX) 

AFUNC=  (  1  ./CALPH*«M  IX  )*ALPHA 
GO  TO  26 


FIGURE  B.2.  COMPUTER  PROGRAM  FOR  HUMAN 
RELIABILITY  PARAMETERS 
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2*  AFUNC=ALPHA 
GO  TO  2' 

2b  AFUNOCALPH»*(Kl-IX>  *ALPHA 
26  PSUM*BFUNC*AFUNC*PSUM 
11  CONTINUE 

20  PROD  ={PSUM  ♦CSTNT)«*N(I)  *P«0U 

2  CONTINUE 

WHITE  (61 1 106)  ALPHA* BETA* PROD 
IF  (PMAX-PHOO) 31*32*32 

31  PMAXaPRQO 

AL*ALPHA 

bL»BElA 

32  bt T A=HET A^H INC 

IF (BETA-BFIN) ld»18*3b 
3b  aLPHAsALPHA*AINC 

IF (ALPHA-AF IN) 33*33*36 
3b  WH1 TE (61  *  109)  PMAX,AL*bL 
bO  TO  1 
40  STOP 

100  FORMAT (I10fF10.2) 

101  Format (6F 10.2) 

102  FORMAT (2110) 

103  FORMAT (10F5.2) 

104  FORMAT  (lFtl*5X*l  IF)  INPUT  OAT A ! // 1  OX . 1 3HPROBAH IL I T Y  =,FS.2/10X*4HNK  = 

l  *  1 3//  1  bX  *  1 3FiK  ( 1 )  N  ( I )  ✓ ) 

10b  FORMAT </lbX»llHG(XT>  TAdLE/) 

106  FORMAT  ( / 1  OX  *F'4 .2  *  1 8H  <  ALPHA  <  *F4.2*bX,  14HINCREMENTS  Of  » 

IF  4 . 2 ) 

107  FORMAT (1H1*13X*26HALPHA  BETA  F  VALUE/) 

10H  FORMAT (9X*2FP.2»F 12.7) 

104  FORMAT «//10X* WHMAXIMUM  VALUE  OF  *F10.7*11H  AT  AL °HA  =*F5.2»8H*  BE 
1 1 a  *«:  5.2) 

110  FOHMAI  ( 7 X . 2  1 1 0 ) 

111  FORMAT  (8X.  U0.F7, 2) 

112  FORMAT  (/10X.F4. 2*  1BFI  <  BETA  S  *F4.2*bX*  14HINCHEMENTS  OF  * 

IF  4  •  2 ) 

tNU 

3200  FORTRAN  UIAGNOST1C  RESULTS  -  FOR  TIMMS 


NULL  SfATtMtNT  NUMHENS 
20 

LOAD*  b6 
RUN*  30 


FIGURE  B .  2  (Cont) 
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APPENDIX  C 

COMPUTER  PROGRAM  FOR  READINESS 
RELIABILITY  CALCULATION 


C.  1  This  appendix  presents  the  flow  logic  and  computer  pregram 

developed  to  calculate  ERU  readiness  reliability. 


C-l 


rr\ 


PF  -  PI(J)  J=1 ,  NUMAX 


PI(1)  [l-G(t)]  P(l,l) 


For  Nu  =2 ,  NUMAX  p  Nu_x 

P(l,Nu)  =  [l-G(t3*ALPHA*|^l-jLi  P(j,Nu-l)*PI(Nu) 

d/t  m  ^  - BETA[l-G(Jxtj}*P(  -1 ,  Nu-l)*PI(Nu~ 

(L  )_  (i-G(ij-i")*t))  J=2Nu 


Punch 

P(J#Nu)s 


For  each  Nu=l ,  NUMAX 


f  Nu  I  Nu 

PRTO(Nu)  =  ALPHA*I  1-^  P(J, Nu)+BETA*J?1P(J, Nu) 


Punch 

PRTO(Nu)s 


NUMAX 

£  P(J,  NUMAX)  -  SUM  1 


J=1 

NUMAX 

E 

j=i 


[l-G(Jnt)] 


-  SUM  2 


FIGURE  C.  I  (Cent) 
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£l-G(TAU)]  *  ALPHA*  (1-SUM  1)+  beta*  SUM  2  —  PR 


1 

PR*PF  -♦  PRPF 
PRODPF*PF  -»  PRODPF 
PRODPR*PR  -♦  PRODPR 
PROD* PRPF  -»  PROD 

- T 

PUNCH 

I,  PROD ,  PRODPF ,  PRODPR 
PRPF,PF,PR 

.  r  ~ 

i  + 1  -i 


I 


FIGURE  C.l  (Cont) 
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0  RELIABILITY  MARK  48  COOK-ZUSMAN  ORL-l/P  JOB  NO.  280  MAv  5,  1965 
DIMENSION  PL  I T ( 20 ) , I POW ( 20 ) , I PFCOD ( 20 ) . TO ( 20 ) ♦DFLP 1 ( 2  0  > ,DELP2(20)  ♦ 
lPI(’O) ,P( 10.10) *  PR  T0 ( 10) 

Gm»l.-EXPE(-DEL1*T**DEL2> 

11  READ  100.NTERU.NTIM1 ,NTIM2 
PUNCH200 ♦NTERU.NT I  Mi ,NTTM? 

PROD*'  1  • 

PRODPR-1 . 

PRODPF  a  1 » 

DO  6  I  *  1 .NTERU 

READ  101 .T, IGCOD.DEL1 , DEL  2 . T AU . NUMAX , ALPHA , BE T A 
PUNCH201 . T , IGCOD.DEL 1 . DEL  2. TAU. NUMAX. ALPHA, RET A 
READ  102, C I PECOD ( J ) , TC ( J \ ,DELP 1 ( J > .D^L P2 ( J ) , J* 1 , NT  I m l j 
PUNCH202  » ( I PECOD ( J)  , TC ( J  > .D^LP 1 f J > ,nrLP2 ( J ) ♦ J= 1 , NT  I m 1  ) 

READ  10’,(PLTT(J) , I POW  f  J ) ,  J  = 1  »NT I M? ) 

PUNCH'’ 33,  (  PL  I  T  (  J  )  ,  I  POW  (  J  )  »  J*  1  ,NTIM2) 

PE*  1  . 

DO  8  J* 1 .NT IM1 

8  PE«PE*EXPF ( -DELE  1 ( J ) #TC ' J > **DELP2 ( J  )  ) 

DO  9  J« 1 » NT  I  M2 

9  PE»PF#PLIT( J)**f POW( J) 

DO  10  J*1 , NUMAX 

10  PI ( J)«PE 

P(1 ,  .  ) * (1 .-G(T ) 1 *P I (1  ) 

DO  4  NU* 2. NUMAX 
SUM-0. 

NUM1-NU-1 
DO  3  J«1,NUM1 

3  SUM*  SUM  +  P  (  J  .NtJMl  ) 

P  C 1  vNU)  *  (1  .-G«  T  )  )  .alpha*  (  I  .  -<u;m  )  *oy  f  nu  ) 

DO  4  J-2.NU 
XJ*  J 

4  P(J. NU  >  * ( 8ETA* (1 #-G ( X J*T ) )*D(j-i ,Nu-l  )  *  p  1  ( N  U ) )/( 1 • - G (  (XJ-1 . )*T)  ) 
PUNCH  305 

DO  53  NU*1 .NUMAX 

53  PUNCH  30?  ,  I ,NU, ( P ( J.NU) . J*1 ,NU) 

DO  1 2  NU* 1 .NUMAX 
SUM*", 

do  n  J*1  » NU 

13  SUM*SUM*P( J.NU) 

12  PRTO(  NU  )  .ALPHA*  (  1  ,-SUM  >  ♦8fr  T  A  *  SUM 
PUNCH  304,  (PR  TO <Nu>  *  NU  *  1  .NUMAX  ) 

SUM1 -0. 

SUM?  *  0  . 

DO  2  J* 1  .NUMAX 

SUM1 -SUMi*P( j, NUMAX ) 

XJ- J 

X JT-X J*T 


FIGURE  C  .  ?  .  COM!  UTER  PROGRAM  FOR 
ERU  READINESS  RELIABILITY 
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2  $UM2»!1.-G!XJT+TAU) ) *P ( J »NUMAX ) / (1 ,-G ( X JT ) )+SUM2 
PR  *{1*-G(TAU) )# ALPHA*} 1.-SUM1 )+BETA*SUM2 
TERM»PR#PF 
PRODPF*PRODPF#PF 
PRODPP«PROPPR»PR 
PR0D«PR0D*?FRM 

6  PUNCH  303*1  *PROD»PRODPF*PRODPR*TFRMtPF,PR 
GO  TO  II 

100  FORMAT (31101 

101  FORMAT!El0.0»Il0*3El0.0*  !10*2E10.0) 

102  FORMAT ( I10*3E10.0) 

103  FORMAT!3(E10„0*!10! 1 

200  FORMAT ( 1 3H1 RELCGMP  MX48/ 

1  7HONTERU=* 15*  7H  NT!M1*,!5*  7H  NTIM2»*I5//) 

201  FORMAT! F16.8* I  3  * 2E 1 6 . 8 , F 1 6 . 8 / 1  OX  .  !5*2Elf>,8  5 

202  FORMAT (!3»F16«8*2F16*8) 

203  FORMAT (3(F16*8*T3)  ) 

302  FORMAT!  4H  ERU*I4,4H  NO* * ! 4 *4E16 .8/ ( 1 6X *4E 16 .8  )5 

303  FORMAT!  4H0ERU  t  I A  *  £H  PROO= . F 1 1 . 8 ♦ 8H  PRODPF »  , F 1 1 . 8 » 8H  PRODPR=  »  FI  1 . 8 
1/8X*6H  PFPR*  *F1 1 .8  »4H  PF  =  , FI  5 . 8  * 4H  PRa»F15.8//> 

304  FORMAT! lOHOPRTO! NU>=*4E16.8/ { 10X*4E16.8n 

305  FORMAT!  1HO « 1 5X 1 8HP ( I  * NU > = / ) 

END 


FIGURE  C.  2  (Cont) 
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APPENDIX  D 

PILOT  TEST  RESULTS  OF  CALCULATION  OF  ESTIMATES 
OF  HUMAN  RELIABILITY  PARAMETERS 


D.  1  The  input  symbols  used  in  this  appendix  are  defined  in 
Appendix  A. 

*  D.2  Common  to  all  test  cases  is  the  following  failure  distribution 
and  value  for  rr: 

w  =  0,95  . 

D.3  The  data  shown  in  Table  D.  1  has  been  used  as  input  for  each 
of  the  four  cases  shown  in  Table  D.?. 

* 

TABLE  D .  1 

;  ASSUMED  EQUIPMENT  FAILURE  DISTRIBUTION 

l 

i 

1 

I 

! 

A 

f 


No.  of  Preventive 
Maintenance  +  1 

Probability  of  Equipment  Failure  Before 
That  Maintenance,  G(xt) 

1 

0.26 

2 

0.45 

3 

0.59 

4 

0.70 

5 

0.77 

6 

0.84 

7 

0.88 

8 

0.91 

9 

0.94 

10 

0.95 
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TABLE  D .  2 


INPUT  DATA  AND  RESULTS  FOR  PILOT  TEST  CASES 


Case 

NK 

ki 

...  Ni 

Z  Ni 

Estimate  of  ft 

1 

3 

1,2,3 

2,2,1 

5 

G  .98 

0.33 

2 

5 

1,2, 3, 4, 5 

1,1, 1.1,  1 

5 

0.57 

0.63 

3 

5 

3, 4, 5, 6, 7 

1,1. 1.1,  1 

5 

0.31 

0.78 

4 

3 

8,9,10 

1,2,2 

5 

0.16 

1.00 

D.4  The  computer  printout  of  the  input  data  and  results  for  the 
mathematical  formulation  of  the  human  reliability  maintenance  parameter 
estimates  are  presented  as  Figure  D.l  on  the  following  pages.  The  pro¬ 
gram  computes  a  probability  {F  value)  that  a  set  of  a  and  8  values  will 
produce  the  corrective  maintenance  experience  reflected  in  the  rest  of 
the  input  data.  It  is  more  likely  that  the  sample  of  corrective  maintenance 
resulted  from  tnat  set  of  «  and  8  values  with  the  highest  probability  than 
from  any  other  set.  The  maximum  likelihood  estimate  of  ry  and  B  is 
therefore  the  set  of  o  and  S  which  has  the  highest  probability  calculated 
for  it . 
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APPENDIX  E 


METHODOLOGY  USED  TO  EVALUATE  HYPOTHESES 


E.  I  The  primary  study  objective  is  to  identify  one  or  a  combination  of 
personnel  characteristics  which,  when  optimally  weighted  under  the  least- 
squares  criterion  for  best  fit,  will  produce  significant  gross  and  multiple 
correlations  with  the  equipment  parameters. 

E.  2  Special  runs  E952A  and  E987A  furnished  by  Pers  19  for  this  research 
task  contained  personnel  characteristics  for  each  of  the  men  on  aboard  the 
selected  ships  for  each  3-month  interval.  Averages  of  the  personnel  char¬ 
acteristics  for  the  men  in  ratings  ET,  RD,  and  ST  were  obtained. 

E.  3  Special  reports  entitled  "Maintenance  History  Records"  (MDC-5) 
were  obtained  from  the  Maintenance  Support  Office,  Mechanicsburg,  Penn¬ 
sylvania,  which  contained  a  chronological  history  of  the  malfunctions  and 
repairs  by  ship  for  each  equipment.  From  these  reports,  a  manual  tabulation 
of  the  total  number  of  malfunctions  and  maintenance  (repair)  times  was  com¬ 
pleted  by  ship  and  by  3-month  period. 

E.  4  At  this  point,  frequency  distributions  of  the  variables  provided 
rough  histograms  so  that  the  normality  of  the  variables  could  be  checked. 

In  many  cases,  the  distributions  were, badly  skewed;  consequently,  all 
raw  data  were  normally  transformed-^  so  that  correlations  would  not  be 
spuriously  inflated. 
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E.5  Selected  scatter  diagrams  of  the  transformed  data  for  each  of  the 

equipment  parameters  with  each  of  the  personnel  parameters  failed  to  dis¬ 
prove  the  assumption  of  linearity  on  which  multiple  correlation  theory  is 
based. 


E.6  After  computer  calculations  were^made  for  all  cases,  gross  (zero 

order)  correlations  and  the  highest  multiple  correlation  coefficients  for  each 
number  of  predictor  variables  included  in  an  analysis  were  tabulated. 

E.7  Adjustments  were  made  in  these  coefficients  in  order  to  avoid 

consistently  overestimating  the  closeness  of  the  relationship  because  of 
small  sample  size.  Formulas-*—'  used  were 


where 


n  =  sample  size 
~t  =  adjusted  value  of  r  ; 

x.y 


and  pa 

1.23. ..k  =1  - 


where  k 

n 
m 


=  number  of  independent  variables 
=  sample  size 

=  degrees  of  freedom  (k  +  1) 

-  adjusted  value  of  R3  , 

1.23. ..k. 


(E.l) 


(E.2) 


E.8  The  adjusted  gross  correlations  which  are  significantly  different 

from  zero  were  identified  by  use  of  a  table  of  critical  values  of  the  correla¬ 
tion  coefficient  at  the  0.05  level  of  significance.  Such  a  a  table  is  found  in 
many  standard  texts.  To  determine  if  adjusted  multiple  correlations  were 
significantly  different  from  zero,  the  follc-wing  F  ratio  was  calculated. 


r 


f 


; 

^  1 
where  R  =  adjusted  multiple  correlation  j 

n  =  sample  size  5 

«/ 

k  =  number  of  predictors 

If  Fj-est>?  05  y  n-k-1  '  corre^ation  is  significant. 


E.9  To  test  whether  or  not  the  addition  of  another  personnel  variable 

would  increase  the  multiple  correlation  significantly,  another  F  ratio  was 
used. 


where 


test 


-(S*)  (eH 


(E.4) 


=  adjusted  multiple  correlation  with  larger  number  of 
personnel  variables 

Ra  =  adjusted  multiple  correlation  with  .smaller  number  of 
personnel  variables 

kx  =  larger  number  of  personnel  variables 
kj,  =  smaller  number  of  personnel  variables 
n  =  sample  size  . 


If  Ftest  >*’o.05,  mx -ma  ,  n-m-1,  t^',e  additional  personnel  variacle  has  increased 
the  correlation  significantly. 

E.  10  In  practice,  when  one  of  the  multiple  correlations  with  two  predictor 
variables  was  significantly  different  from  zero,  Formula  (E.4)  was  used  to  see 
if  an  additional  variable  would  increase  the  correlation  significantly. 

E.  11  When  it  is  known  that  a  multiple  correlation  is  significantly  different 
from  zero,  it  is  possible  to  calculate  a  correlation  for  which  it  is  95  percent 
certain  that  the  true  correlation  is  greater  than  the  calculated  correlation. 

E.  IE  Ezekielr-^  provides  convenient  graphs  from  which  this  value  can 

be  estimated  at  the  95  percent  confidence  level.  For  example,  if  for  50 
observations  a  multiple  correlation  of  0.62  is  obtained  using  three  personnel 
variables,  from  the  graph  it  can  be  stated  that  there  is  a  certainty  of  95  percent 
that  0.42  is  the  probable  minimum  correlation  in  the  universe. 
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E.  13  When  multiple  correlation  coefficients  are  significantly  dif*erent 
from  zero,  knowledge  of  the  relative  importance  of  the  personnel  character¬ 
istics  is  desirable.  To  obtain  this,  the  £  coefficient  is  calculated.-^ 
Thus,  the  most  influential  personnel  variable  is  indicated  by  the  largest 
absolute  $  vaiue. 
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